OK, I figured out what was causing the problem. Just thought I will leave the answer here in case someone else runs into the same problem.
If the street address is not properly capitalized, it is not read properly as the address!
Here is my XML autolink_test string:
Name: New York Times \n
Email: public@nytimes.com \n
Phone: 212-556-7652 \n
Address: 620 Eighth Avenue New York, NY 10018 \n
Address: 620 Eighth avenue New York, NY 10018 \n
Website: http://www.nytimes.com
The first address shows up correctly as an autolink.
The second one (with a small 'a' in 'avenue') does not show up correctly.
This seems a little strange to me as the google maps website certainly doesn't care about such niceties.
Anyways, so here it is :-)