The U.S. had team-record 16 victories in 2013 and qualified for the World Cup for a seventh time in succession. It was the eighth defeat in 44 matches under Klinsmann.
"Disappointed in the result, but I think we played pretty well," U.S. goalkeeper Tim Howard said before adding: "The year 2013 has been one of our best years ever."
Austria failed to reach Brazil 2014 after being eliminated from a group won by Germany.
The U.S. failed to score for a second straight match after being held to goalless draw by Scotland on Friday.
The trip "was definitely worth it," Klinsmann said. "It was a good 10 days, good to see the players compete in training and these two games. You learn a lot about the guys when they are in a training camp."
After an unimpressive opening 10 minutes in front of 20,200 spectators, Austria created two chances through Martin Harnik. The winger dribbled past DaMarcus Beasley coming in from the left wing but couldn't beat Howard at the near post.
The next minute, Harnik's quick move opened up the American defense but his cross didn't find Janko.
The Americans responded with 20-meter efforts by Jozy Altidore, whose shot went wide, and Aron Johannsson, who forced keeper Robert Almer to a punching save.
"It's been a fun ride, and it's been amazing to be a part of watching this team grow from the loss to Honduras to how we picked ourselves up," Altidore said. "It's been inspiring."
Harnik almost doubled the lead in the 49th with a curling shot past keeper Howard, though TV footage showed the winger was offside while collecting a pass by captain Christian Fuchs.
The United States pressed for an equalizer, but Austria played deep in its own half and allowed hardly any space.
"Over both wings we didn't create enough chances," Klinsmann said.
Altidore came closest but his low shot in the 74th was saved by Almer, and he sent a free kick wide in the closing minute.
"It was important to keep a clean sheet and to score a goal," Austria coach Koller said. "We were playing a top-13 team and you see they played some good football. They definitely had chances but you can't win if you don't score from them."
Koller took his team to a seven-day training camp in Alicante, Spain, last week in order to test some tactical variations.
He changed his team in several positions, giving defender Martin Hinteregger and attacking midfielder Lukas Hinterseer a place in the starting lineup for their first games for Austria.