html5 Video火狐LiUx,html - Firefox and Chrome can't play HTML5 mp4 video

I have an HTML page that contains a MP4 video:

Your browser does not support the video tag.

When I load the page in Firefox, it doesn't show the play control buttons but shows error message "No video with supported format and MIME type found" (see the screenshot below).

So I use this site to test my browser's ability to play HTML5 mp4 video and it can successfully play the test video on that site. My Firefox version is v36 on CentOS Linux. I also tried using Chrome and it can't play it either. I also tried it on Firefox/Chrome on Windows but failed. I then use the Firefox debugger to look at the debug info and I see the following message:

Media resource http://localhost:5000/static/xyz.mp4 could not be decoded.

All candidate resources failed to load. Media load paused.

I then tried playing the xyz.mp4 video on my local machine using Linux's movie player and it plays fine and it also plays fine in Window's media player. But when I use the above HTML5 tag, it doesn't play for all browsers in both Linux and Windows. I also followed Mozilla's online forum to change the browser settings and clear caches but none of them works. So what caused the problem of this simple HTML5 ?