嵌入式下VB.NET窗体操作 DLL

SetWindowPos returns a Boolean value. Also under VB.NET a Long value is 64 bits and not 32 which was the case in previous versions of VB. This is why you are getting an exception. In VB.NET Integer is 32 bits, Short is 16 and Byte is 8.
Also check that all the constants you are passing to SetWindowPos are Integers and not Long values.

I'm also assuming you're trying to initiate fullscreen mode. To do this use the following code (includes p/invoke definitions) :

Imports System.Runtime.InteropServices

<DllImport ("coredll .dll", EntryPoint:="SetForegroundWindow", SetLastError:=True)> _
Private Function SetForegroundWindow( _
ByVal hwnd As IntPtr) As Boolean
End Function

<DllImport ("coredll .dll", EntryPoint:="SetWindowPos ", SetLastError:=True)> _
Private Function SetWindowPos ( _
ByVal hwnd As IntPtr, _
ByVal hWndInsertAfter As IntPtr, _
ByVal X As Integer, _
ByVal Y As Integer, _
ByVal cx As Integer, _
ByVal cy As Integer, _
ByVal uFlags As Integer) As Boolean
End Function

<DllImport ("aygshell.dll", EntryPoint:="SHFullScreen", SetLastError:=True)> _
Private Function SHFullScreen( _
ByVal hwndRequester As IntPtr, _
ByVal dwState As Integer) As Boolean
End Function

Private Const SHFS_HIDETASKBAR As Integer = &H2
Private Const SWP_NOZORDER As Integer = &H4

SetForegroundWindow(hwnd)
SHFullScreen(hwnd, SHFS_HIDETASKBAR)
SetWindowPos (hwnd, IntPtr.Zero, 0, 0, Screen.PrimaryScreen.Bounds.Width, Screen.PrimaryScreen.Bounds.Height, SWP_NOZORDER)