The Basics of the MMAPI for Java Developers->Media Capture

The Basics of the MMAPI for Java Developers

• By Michael Juntao Yuan, Kevin Sharp.
• Sample Chapter is provided courtesy of Addison Wesley Professional.
• Date: Apr 15, 2005.

SaveDiscussPrintE-mail

Article Information

Article Description

In this chapter, you will learn the basics of the MMAPI and its implementation on Nokia devices. After studying the two example applications provided, you should be able to add multimedia playback and audio/video capturing features to your own application.

From the Book

Developing Scalable Series 40 Applications: A Guide for Java Developers

$44.99 (Save 10%)

Media Capture

For devices equipped with voice recorders or digital cameras, the MMAPI Player class can be used to capture audio clips or snap digital pictures. The media capturing support is currently available only on Series 60 devices and will come soon on future Series 40 devices. In this section, we use a Series 60–based multimedia blog application to demonstrate the media capture mechanisms. The mobile client allows users to take a picture, record a short audio clip, and then send both media files with a text note to a server. The server publishes the entries with timestamps to an HTML Web page in the chronological order. Any user can then view the blog entries from any xHTML-compatible browser on a PC or on a phone. The process is illustrated in Figure 9-7.

Figure 9-7 The multimedia blog.

Note

Blog is the abbreviated term for "Web log." It is a personal Web page that records and publishes a person's daily journal or thoughts.

Capture Image

We can instantiate a video capture Player object by passing the URI locator capture://video to the Manager.createPlayer() factory method. Then, we can get a VideoControl control from the player and display the video UI window to the LCD using the techniques we discussed in the video playback section. Once the player is started, the user sees the live motion video of the camera view in the VideoControl UI. We can then call the VideoControl.getSnapshot() method to take a snapshot and store the photo to a byte array. The argument we pass to the getSnapshot() method determines the type and the dimensions of the resultant image. The following list is specific to Nokia 6600 and 3650 devices.

• The getSnapshot(null) call captures a 160 by 120 PNG image.
• The getSnapshot("encoding=jpeg") call captures a 160 by 120 JPEG image.
• The getSnapshot("width=320&height=240") call captures a 320 by 240 PNG image.
• The getSnapshot("encoding=jpeg&width=320&height=240") call captures a 320 by 240 JPEG image.

Three encoding values, png, jpeg, and bmp, are supported. The gif encoding is recently added to the Nokia 6600 maintenance release. If values are specified for width and height, both must be specified. If the requested dimensions are different from 160 by 120, the image is scaled to the requested width and height. If the aspect ratio requested does not match 4:3 (the default aspect ratio), the resulting image could be distorted. The maximum image size that can be captured depends on the free heap memory available.

Note

The Series 60 MIDP concept emulator 0.3.1 only supports capturing 160 by 120 PNG images. The emulator uses a stock still image to emulate the camera viewfinder (Figure 9-8).

Figure 9-8 Camera viewfinder on the Series 60 emulator.

Note

For security reasons, some of the media capture APIs are restricted. On Nokia 3650, the user has to give explicit permission to the getSnapshot() method call. On Nokia 6600 and other MIDP 2.0 devices, we can sign the MIDlet suite and place it under a trusted domain to suppress security prompts that would otherwise break into the screen.

The following code shows how to develop a photo capture screen CameraView. When the user hits the action key or the capture menu, the camera takes a 320 by 240 JPEG snapshot image.

public class CameraView extends Form
            implements CommandListener {

  private Command capture;
  private Command skip;

  private Player player = null;
  private VideoControl video = null;

  public CameraView () {
    super ("Take a picture");

    showCamera ();

    capture = new Command ("Capture", Command.OK, 1);
    skip = new Command ("Skip", Command.CANCEL, 1);
    addCommand (capture);
    addCommand (skip);
    setCommandListener (this);
  }

  public void commandAction (Command c, Displayable d) {
    if (c == capture) {
      capture ();
      BlogClient.showPhotoPreview ();
  } else if (c == skip) {
    player.close ();
    BlogClient.showAudioRecorder ();

  }
}

private void showCamera () {
  try {
    player = Manager.createPlayer("capture://video");
    player.realize();

    // Add the video playback window (item)
    video = (VideoControl) player.getControl(
                                        "VideoControl");
    Item item = (Item) video.initDisplayMode(
        GUIControl.USE_GUI_PRIMITIVE, null);
    item.setLayout(Item.LAYOUT_CENTER |
                    Item.LAYOUT_NEWLINE_AFTER);
    append (item);
    // Add a caption
    StringItem s = new StringItem ("", "View finder");
    s.setLayout(Item.LAYOUT_CENTER);
    append (s);

    player.start();

  } catch (Exception e) {
    e.printStackTrace ();
  }
}

private void capture () {
  try {
    // PNG, 160x120
    // BlogClient.photoData = video.getSnapshot(null);
    //      OR
    // BlogClient.photoData = video.getSnapshot(
    //     "encoding=png&width=160&height=120");

    // BlogClient.photoPreview = BlogClient.photoData;
    // BlogClient.photoType = "png";

    byte [] tmp = video.getSnapshot(
        "encoding=jpeg&width=320&height=240");
    BlogClient.photoPreview =
           BlogClient.createPreview(
              Image.createImage(tmp, 0, tmp.length));
      BlogClient.photoData = tmp;
      BlogClient.photoType = "jpg";

      player.stop ();
      player.close ();

    } catch (Exception e) {
      e.printStackTrace ();
      BlogClient.showAlert ("Error", e.getMessage ());
    }
  }
}

The BlogClient.createPreview() method resizes the snapshot to a smaller preview image, which fits into the device screen. The method first creates an empty image of the preview size. It then populates the preview image pixel by pixel by sampling the original image. This method creates an approximate thumbnail without parsing the original JPEG image.

// Scale down the image by skipping pixels
public static Image createPreview (Image image) {
  int sw = image.getWidth();
  int sh = image.getHeight();

  int pw = 160;
  int ph = pw * sh / sw;

  Image temp = Image.createImage(pw, ph);
  Graphics g = temp.getGraphics();

  for (int y = 0; y < ph; y++) {
    for (int x = 0; x < pw; x++) {
      g.setClip(x, y, 1, 1);
      int dx = x * sw / pw;
      int dy = y * sh / ph;
      g.drawImage(image, x - dx, y - dy,
          Graphics.LEFT | Graphics.TOP);
    }
  }

  Image preview = Image.createImage(temp);
  return preview;
}

The PhotoReview class shows the review image and asks the user whether to continue.

public class PhotoPreview extends Form
              implements CommandListener {

  private Command cancel;
  private Command next;

  public PhotoPreview () {
    super ("Photo Preview");
    cancel = new Command ("Cancel", Command.CANCEL, 1);
    next = new Command ("Next", Command.OK, 1);
    addCommand (cancel);
    addCommand (next);
    setCommandListener (this);

    append (new ImageItem (
        "Image size is " + BlogClient.photoData.length,
        BlogClient.photoPreview,
        ImageItem.LAYOUT_CENTER, "image"));
  }

  public void commandAction (Command c, Displayable d) {
    if (c == cancel) {
      BlogClient.initSession();
      BlogClient.showCamera ();

    } else if (c == next) {
      BlogClient.showAudioRecorder();
    }
  }
}

Capture Audio

After the user is satisfied with the photo, she can record an audio message. The audio capture player is instantiated using the capture://audio URI locator. Arguments to this string are used to specify audio bitrate and sampling depth. For example, the capture://audio?rate=8000&bits=16 URI locator string initiates a Player object and captures 16-bit sound at 8,000 bits per second. Once we have the Player instance, we can get a RecordControl control and use its setRecordStream() method to assign an OutputStream to this control. The default audio output format is the wav format in Nokia 6600 and 3650 devices. The RecordControl starts recording once the player enters the started state and stops once the player is stopped.

Note

Using the RecordControl in a video capture player, we can theoretically capture full-motion video clips from the camera. However, currently no video encoding format is supported on any Nokia device.

The source code for the AudioRecorder class is listed below. When the user stops the audio capturing player, the program writes out the recorded audio data to a byte array, audioData.

public class AudioRecorder extends Form
              implements CommandListener {

  private Command skip;
  private Command start;
  private Command stop;

  private Player player = null;
  private RecordControl recordcontrol = null;
  private ByteArrayOutputStream output = null;


  public AudioRecorder () {
    super ("Audio Recorder");

    skip = new Command ("Skip", Command.CANCEL, 1);
    start = new Command ("Start", Command.OK, 1);
    stop = new Command ("Stop", Command.SCREEN, 1);
    addCommand (skip);
    addCommand (start);
    addCommand (stop);
    setCommandListener (this);

    append ("Use the Start menu to start recording. " +
        "Use the Stop menu to stop recording");
    try {
      player = Manager.createPlayer(
          "capture://audio?rate=8000&bits=16");
      player.realize ();
      recordcontrol =
  (RecordControl) player.getControl("RecordControl");
      output = new ByteArrayOutputStream ();
      recordcontrol.setRecordStream(output);

    } catch (Exception e) {
      e.printStackTrace ();
    }
  }

  public void commandAction (Command c, Displayable d) {
    if (c == skip) {
      BlogClient.showMessageForm ();

    } else if (c == start) {
      try {
        recordcontrol.startRecord();
        player.start();
      } catch (Exception e) {
        e.printStackTrace ();
        BlogClient.showAlert ("Error",
            "Cannot start the player. " +
            "Maybe audio recoding is not supported " +
            "on this device. Please skip this step");
      }

    } else if (c == stop) {
      try {
        recordcontrol.commit ();
        player.close();
      } catch (Exception e) {
        e.printStackTrace ();
        BlogClient.showAlert ("Error",
            "Cannot stop the player");
      }
      BlogClient.audioData = output.toByteArray();
      BlogClient.showMessageForm ();
    }
  }
}

Submit Blog Entries

After we have gathered the snapshot, audio recording, and short text message data, we can submit it to the server. The code below illustrates how it works. The complete source code with multithread support is available in the BlogClient class.

HttpConnection conn = null;
DataInputStream din = null;
DataOutputStream dout = null;

conn = (HttpConnection) Connector.open(url);
conn.setRequestMethod(HttpConnection.POST);

dout = conn.openDataOutputStream ();
dout.writeInt (PHOTO_AUDIO);
dout.writeUTF (message);
dout.writeUTF (photoType);
dout.writeInt (photoData.length);
dout.write (photoData, 0, photoData.length);
dout.writeInt (audioData.length);
dout.write (audioData, 0, audioData.length);

dout.flush ();
dout.close ();

The Blog Servlet

On the server side, the blog servlet intercepts the HTTP POST data in the doPost() method. It stores image and audio data into separate files and appends links to an HTML master file. The doGet() method is invoked when a client accesses the servlet directly from a browser. It assembles and returns an HTML page with up-to-date blog entries. The full source code of the servlet is listed below for your reference.

public class BlogServlet extends HttpServlet {
  private static int NO_OBJECT = 0;
  private static int PHOTO_ONLY = 1;
  private static int AUDIO_ONLY = 2;
  private static int PHOTO_AUDIO = 3;

  private static int SUCCESS = 1;
  private static int FAILURE = 2;

  private static String fileroot = null;
  private static String webroot = null;

  public void init() throws ServletException {
    InputStream in =
        getClass().getResourceAsStream("/conf.prop");
    Properties prop = new Properties ();
    try {
      prop.load (in);
      fileroot = prop.getProperty("Fileroot");
      webroot = prop.getProperty("Webroot");
    } catch (Exception e) {
      e.printStackTrace ();
      fileroot = "/root/BlogServer/content/";
      webroot = "/BlogServer/content/";
    }
  }

public void doGet(HttpServletRequest request,
                   HttpServletResponse response)
            throws ServletException, IOException {
  response.setContentType("text/html");
  PrintWriter out = response.getWriter();

  out.println(readTextFile("header.html"));
  out.println(readTextFile("entries.html"));
  out.println(readTextFile("footer.html"));

  out.flush ();
  out.close ();
}

public void doPost(HttpServletRequest request,
                     HttpServletResponse response)
             throws ServletException, IOException {

  response.setContentType("application/binary");

  InputStream in = request.getInputStream();
  OutputStream out = response.getOutputStream();
  DataInputStream din = new DataInputStream(in);
  DataOutputStream dout = new DataOutputStream(out);

  String current =
      Long.toString((new Date()).getTime());
  String body = "<b>Posted at</b>: " +
      (new Date()).toString() + "<br/>";
  String filename;

  // Get the opcode
  int opcode = din.readInt();
  // Get the message body
  body += din.readUTF();
  body += "<br/>";

  if (opcode == NO_OBJECT) {
    // Save the message body
    saveMessage (body);
    dout.writeInt(SUCCESS);

  } else if (opcode == PHOTO_ONLY) {
    // Get photo data
    String photoType = din.readUTF ();
    byte [] photoData = receiveObject (din);

    // Save the photo data
    filename = current + "." + photoType;
    saveObject (photoData, filename);
    body = body + photoHtml(filename);

    // Save the message body
    saveMessage (body);
    dout.writeInt(SUCCESS);

  } else if (opcode == AUDIO_ONLY) {
    // Get the audio data
    byte [] audioData = receiveObject (din);

    // Save the audio data in a file
    filename = current + ".wav";
    saveObject (audioData, filename);
    body = body + audioHtml(filename);

    // Save the message body
    saveMessage (body);
    dout.writeInt(SUCCESS);

  } else if (opcode == PHOTO_AUDIO) {
    // Get the photo data
    String photoType = din.readUTF ();
    byte [] photoData = receiveObject (din);
    // Get the audio data
    byte [] audioData = receiveObject (din);

    // Save the photo data in a file
    filename = current + "." + photoType;
    saveObject (photoData, filename);
    body = body + photoHtml(filename);

    // Save the audio data in a file
    filename = current + ".wav";
    saveObject (audioData, filename);
    body = body + audioHtml(filename);

    // Save the message body
    saveMessage (body);
    dout.writeInt(SUCCESS);

  } else {
    dout.writeInt(FAILURE);
  }
  dout.flush();
  dout.close();
  din.close();
  in.close();
  out.close();
}

private byte [] receiveObject (DataInputStream din)
                                   throws IOException {
  int length = din.readInt();
  byte [] buf = new byte [length];
  din.readFully (buf);
  return buf;
}

private void saveObject (byte [] data, String name)
                                    throws IOException {
  FileOutputStream fout =
      new FileOutputStream(fileroot + name);
  fout.write(data, 0, data.length);
  fout.flush ();
  fout.close ();
}

private void saveMessage (String body)
                        throws IOException {
  body = body + "<hr/>";
  FileWriter writer =
    new FileWriter (fileroot + "entries.html", true);
  writer.write(body, 0, body.length());
  writer.flush ();
  writer.close ();
}

private String photoHtml (String filename) {
  return "<img src=/"" + webroot + filename +
          "/"/>" + "<br/>";
}

private String audioHtml (String filename) {
  return "<a href=/"" + webroot + filename +
    "/">" + "Audio file (wav format)</a>" + "<br/>";
}

private String readTextFile (String name)
                         throws IOException {
  StringBuffer result = new StringBuffer ();
  FileReader reader =
       new FileReader (fileroot + name);

    char [] buf = new char [256];
    int i = 0;
    while ( (i = reader.read(buf)) != -1 ) {
      result.append(buf, 0, i);
    }
    reader.close();
    return result.toString ();
  }
}

5. Summary | Next Section Previous SectionThe Basics of the MMAPI for Java Developers