In general, the most reliable way of getting two phones talking to each other is using C2DM (or GCM, now). Perhaps you can have the devices exchange their IP addresses using C2DM (which may get you nothing at all, btw, if its an address behind a router or firewall with blocked ports). You could also use SMS depending on the needs of your apps.
Usually there'd be a server of yours involved--additional development--but for short messages, I don't see why one device cannot directly post via C2DM to another device. (This is an untested statement, beware.)