Native Extensions For Microsoft Silverlight

Resource Page Description While Silverlight 4 supports accessing COM automation components from elevated trust OOB applications, many Windows platform features are currently not available through COM automation. This makes them inaccessible to such Silverlight OOB apps. Native Extensions for Microsoft Silverlight(NESL) is an effort to incrementally build a library of components that expose select Windows 7 features through COM automation, making them easily available to Silverlight 4 OOB applications running with elevated trust. The current version of NESL provides access to Windows 7 features like Sensors, Portable Devices, Speech, Taskbar and more. NESL is made up of a set of COM automation based runtime libraries, Silverlight wrapper libraries usable from Silverlight 4 OOB applications, sample applications with source, API documentation, and a developer's guide. UPDATE 06/15/2011 - NESL version 2.1 (bug fix update) is now available at http://archive.msdn.microsoft.com/nesl/Release/ProjectReleases.aspx?ReleaseId=5519 No new features, bug fixes only Incremental source update to NESL v2.0, requires NESL v2.0 source bits installed on developer machines prior to installing the fix update For developer machines: 1. If you do not have NESL v2.0 source bits installed, please install it prior to applying the update. Otherwise skip this step. 2. Download, unzip and install the 2.1 runtime from the NESLRuntime2.1_fixes.zip. 3. Download and unzip the contents from the NESLSource2.1_fixes.zip in the same location where you have installed the source from your version 2.0 install. This should overwrite only one of your Silverlight projects – namely Microsoft.Silverlight.Windows. 4. If you are deploying NESL “in-app” then change the calls to Installer.CheckNESLInstalled() in your code to reflect the new version number like so: Installer.CheckNESLInstalled(2,1) For deployment If you are performing “in-app” deployment, make sure you package the latest NESLSetup.msi file (obtained from step 2 above) with your XAP. UPDATE 03/03/2011 - NESL version 2.0 is now available at http://archive.msdn.microsoft.com/nesl/Release/ProjectReleases.aspx?ReleaseId=5519 New Features Touch Features Gesture support Manipulation processing support Inertia processing support Touch hardware information Windows 7 Notification Area (System Tray) support Add/Update/Remove notification icons to the notification area Add/Remove notification information balloons Respond to notification and notification balloon events Add/Update/Remove context menus to notification icons Respond to context menu selections “Minimize to tray”/”Restore from tray”   “Single application instance” pattern support Allow only a single instance of the application to run at any time Be notified when a subsequent launch is attempted Aids in applications that add themselves to the system tray, and hence needs to maintain a single running instance. New feature demos illustrating for Local Encoding, Portable Devices, Touch and Sensors Full Developer Documentation and updated API Guide Bug Fixes UPDATE 02/08/2011 - NESL version 2.0 Preview Build 2 now available http://archive.msdn.microsoft.com/nesl/Release/ProjectReleases.aspx?ReleaseId=5445 New Features Touch Features Gesture support Manipulation processing support Inertia processing support Touch hardware information Windows 7 Notification Area (System Tray) support Add/Update/Remove notification icons to the notification area Add/Remove notification information balloons Respond to notification and notification balloon events Add/Update/Remove context menus to notification icons Respond to context menu selections “Minimize to tray”/”Restore from tray”   “Single application instance” pattern support Allow only a single instance of the application to run at any time Be notified when a subsequent launch is attempted Aids in applications that add themselves to the system tray, and hence needs to maintain a single running instance. New feature demos illustrating above features UPDATE 1/20/2011 - NESL version 2.0 Preview Build now available http://archive.msdn.microsoft.com/nesl/Release/ProjectReleases.aspx?ReleaseId=5368 New Features Touch Features Gesture support Manipulation processing support Inertia processing support Touch hardware information New Feature Demos for Local Encoding, Portable Devices, Touch and Sensors Bug fixes UPDATE 1/13/2011 - Upcoming MSDN Webcasts on NESL  1/18/2011 10:00 AM        MSDN Webcast: Native Extensions for Microsoft Silverlight (NESL) (Part 1 of 2) https://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032473313&Culture=en-US 1/25/2011 10:00 AM MSDN Webcast: Native Extensions for Microsoft Silverlight (NESL) (Part 2 of 2) https://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032473316&Culture=en-US What APIs are covered in NESL v1 ? Windows Sensor API Integrate sensors in you Silverlight app – Accelerometers, Light sensors, compasses, gyroscopes etc. Microsoft Speech API Text to Speech (Speech Synthesis) Speech to Text (Speech Recognition) Windows Portable Devices API Enumerate and access content from connected portable devices (cameras, phones, music players, scanners etc.) Windows 7 Taskbar Integration Jump lists, overlay icons, overlay progress display, Thumbnail toolbars etc. Webcam Local Encoding and Window Capture Encode webcam capture into mp4 (H.264/AAC-LC) media Capture user interface interaction as video Window Message Interception Intercept and act on Window messages sent to the Silverlight OOB host window Platform Requirements Windows 7 32-bit or 64-bit operating system Silverlight 4 application running out-of-browser with elevated trust Sample Applications to illustrate API usage Video Travel Journal - Capture webcam audio/video locally as an MP4 file. Video - H.264 baseline profile, Audio - AAC-LC Overlay with Bing map locations, titling, tickers, pen annotations, and images from a connected camera or USB storage. Upload to Facebook Windows 7 Taskbar Integrated           Where's Teddy - An animated children's book Concept application that narrates a children's story Text-to-speech based narration Voice Commands to navigate story parts Light sensor integration to control scene lighting Accelerometer integration to shake objects           Touch support in both samples NOTE: Facebook upload functionality in the Travel Journal sample: This functionallity is provided as a sample of how you might be able to achieve similar functionality in your own application. However there are a few steps required on your part to activate this functionality in the sample application itself, without which this functionality will not work in the sample. We have included some helper classes that will aid you in getting this up and running. Please follow the steps outlined below to activate this feature in the Travel Journal sample application: Step 1 - Create a new ASP.NET MVC 2 application, or use the provided TravelJournalFacebookConnector application. If you are using the provided application, skip to Step 4. Step 2 - Add the FacebookController.cs file from the Controllers folder. Add the RoutingTable.cs, FacebookVideo.cs and FacebookSettings.cs file to the Models folder. Also add Notify.aspx to the Views folder of the project, in a subfolder named facebook. You will find these files in similarly named folders in the included TravelJournalFacebookConnector application. Step 3 - Add the TravelJournal Silverlight application to the Silverlight Applications properties tab in the newly created ASP.NET MVC application project propery page. Step 4 - Register the application with Facebook. In order to register an application you must be a registered Facebook user. Step 5 - Verify your application with Facebook by supplying a mobile telephone number. Facebook sends a verification code as a text message to your mobile phone which you can then use to verify the application. Step 6 - On your 'My Applications' page you will have access to the details and settings associated with your application. By clicking on 'Edit settings' you can enter your website details, add a logo and a description, etc. Step 7 - In the FacebookSettings (in TravelJournal.Web) class you need to fill in a number of default values that are set in the constructor using the information from you Facebook application (these should be visible on the 'My Applications' page). You must add the following information: Application ID : Required. Maps to AppID in FacebookSettings Application Secret : Required. Maps to AppSecret in FacebookSettings API Key : Required. Maps to ApiKey in FacebookSettings. Note: The Facebook authorization flow requires that Facebook's login page be hosted in a browser control. In Silverlight we are able to use the WebBrowser control to achieve this goal. However security restrictions mean that we must pass data from the authorization process to the Silverlight application by using a script to raise a NotifyScript event on the WebBrowser control. In order for this script to be run it must be sourced from the same domain which the Silverlight application was deployed from. The solution presented in this demo is to deploy a small ASP.NET MVC application that is hosted on the deployment server. In a development environment you will be able to launch this server component as a part of the project however for deployment you will need to meet the following requirements: 1. Ensure that the server component will be hosted on the same domain where the Silverlight 4 OOB application will be deployed from, e.g. if the deployment page is located at http://www.example.com/Deploy.html then the application will navigate to http://www.example.com/facebook/login to begin the Facebook authorization process. 2. Ensure that you have the required components on the host server. This may require installing and/or updating components on the host server. The following are required for this application: .NET 4.0 Framework ASP.NET MVC 2 Framework Sensor Hardware: We tested the sample applications using an externall attached, reference sensor board manufactured by freescale semiconductor. If you need it, you can purchase the sensor board from here - http://www.freescale.com/webapp/sps/site/prodsummary.jsp?code=JMBADGE&tab=BuyParametric_Tab&fromSearch=false&tid=m8TSPJMBADGE2008-B You can download the Sensor Development kit drivers from here - http://archive.msdn.microsoft.com/SensorsAndLocation/Release/ProjectReleases.aspx?ReleaseId=2359 You can also check out this blog entry on how to acquire the drivers and install the reference board on your device - http://blogs.msdn.com/b/gavingear/archive/2009/10/16/great-article-covering-windows-7-sensor-development-kit-driver-install.aspx If you are testing the samples on a Windows 7 device that already has sensors and you want to utilize those, you will need to ensure that appropriate drivers exist for those sensors and that they are recognized properly by Windows 7. To ensure this, please navigate to Control Panel|Location and Other Sensors and ensure that the sensors you are targeting are displayed there. Library Structure Runtime libraries that provide the COM automation access to the underlying Windows APIs. Silverlight wrapper classes around the runtime libraries to be used directly from your applications. What's available for download ? Standalone MSI installer for the native runtime Use to deploy NESL Developer setup Source code for Silverlight wrapper libraries Source code for sample applications API Documentation Developer's Guide to NESL (Coming Soon !!) MSI installer for native runtime What's happening beyond this release ? More features planned Check back soon for announcements !!

Last edited Jun 16 2011 at 4:43 AM by jitghosh, version 27

Comments mscherotter wrote                                  Dec 3 2010 at 6:26 AM                                                              Great Job!                                                      tarlano wrote                                  Dec 4 2010 at 4:32 PM                                                              Fantastic!                                                      AlexanderBykin wrote                                  Dec 10 2010 at 7:35 PM                                                              wow, good job. thanks for this.                                                      JochemKempe wrote                                  Jan 20 2011 at 7:36 AM                                                              Looks very promising. I will certainly have to look into this, first thing tomorrow (I'm Europe based ;-))                                                      SubodhNPushpak wrote                                  Jan 20 2011 at 2:48 PM                                                              Awesome! Finally, 3-screens-and-cloud seems to be promising. Silverlight works on web, Windows Phone 7 and now natively for windows 7. Wish the extensions works for Mac as well.. :D                                                      AxeNaJa wrote                                  Mar 8 2011 at 1:52 AM                                                              good                                                      theresesamith wrote                                  Nov 24 2012 at 11:51 AM                                                              tks for the information. :)                                              Sign in to add a comment