huffman编码是常见的压缩算法之一,
例如我们平常看到的jpeg图片就是用huffman进行熵编码压缩。
我实现了huffman的alchemy版和AS3版两个版本。
纯AS3实现是把C算法完整地移植过去的(使用uint的Array数组模拟char[])
因为huffman算法在网上比较容易找到,我就不详细介绍。
下面只着重记录如何用Alchemy技术把huffman的API暴露到AS3,
然后与纯AS3实现的结果进行比较。
(这里说的huffman算法压缩后头四个字节为解压长度,压缩使用位流)
一、安装Alchemy。
Alchemy的安装方法以前已经说过,不过我是按照官方的做法再做了一次
(稍有不同的是我的alchemy不需要alc-on):
以下以windows xp+cygwin为开发环境(假设已经安装flex sdk和jdk)
1. 在adobe官方网站下载alchemy的cygwin二进制包。
2. 本地或在线安装cygwin,Select packages时选中make,perl、zip(不需要安装gcc,也不需要安装源代码)。
本地安装是cygwin把在线安装的文件缓存在硬盘中,最开始一般都是在线安装。
选择zip时要小心,不要选其它名字带zip的压缩工具。zip程序用于后面生成swc文件。
3. 启动cygwin的批处理文件Cygwin.bat,完成cygwin的全部安装。
4. 把alchemy的二进制文件解压到cygwin安装目录下
5. 在/etc/profile最后加入
export PATH=/cygdrive/D/java/flex_sdk_4.1.0.16076/bin:$PATH
export PATH=/cygdrive/D/java/jdk1.6.0_20/bin:$PATH
export PATH=/alchemy-cygwin-v0.5a/achacks:/alchemy-cygwin-v0.5a/bin:$PATH
export ALCHEMY_HOME=/alchemy-cygwin-v0.5a
export FLEX_HOME=D:/java/flex_sdk_4.1.0.16076
export ASC=C:/cygwin3/alchemy-cygwin-v0.5a/bin/asc.jar
注意:这里jdk、flex sdk的位置和alchemy的目录位置可变;
cygdrive用于转换windows的绝对路径;
冒号用于分割;
另外FLEX_HOME和ASC最好不要用cygdrive转换,因为它用于flex的jar文件定位,
windows下的java可能无法识别路径名而报错;
因为cygwin没有装gcc,所以我直接把achacks目录加入PATH(与官网的安装方法不一样,所以后面不需要alc-on那么麻烦)
把/etc/profile保存为Unix换行(注意:需要用特殊的编辑器编辑,我是使用Notepad2 MOD菜单中的“行末符号->Unix换行”保存)
重新启动cygwin控制台以使环境变量生效。
6. 执行以下命令
$ cd /alchemy-cygwin-v0.5a/
$ ./config
Generating alchemy-setup...
Turning execution bit on for Alchemy binaries...
cygwin warning:
MS-DOS style path detected: C:/cygwin3/alchemy-cygwin-v0.5a/alchemy-setup
Preferred POSIX equivalent is: /alchemy-cygwin-v0.5a/alchemy-setup
CYGWIN environment variable option "nodosfilewarning" turns off this warning.
Consult the user's guide for more details about POSIX paths:
http://cygwin.com/cygwin-ug-net/using.html#using-pathnames
Add "source /alchemy-cygwin-v0.5a/alchemy-setup" to your login script.
"alc-home" takes you to the Alchemy install folder.
"alc-on" puts Alchemy gcc toolchain replacements at the front of your path.
"alc-off" restores original path.
"alc-util" shows you various Alchemy-related environment vars
You need Flash 10 or AIR 1.5 and the Flex 3.2 SDK installed for testing.
7. 测试一下以下命令:
$ explorer .
$ mxmlc -help (如果无法运行,请确保jar的路径是windows的绝对路径,设置FLEX_HOME环境变量)
$ java -help
$ perl --help
$ zip --help
8. 测试gcc是否正常:(不需要alc-on,是因为我已经把alchemy的gcc加入了PATH,而且cygwin中并没有装真正的gcc)
$ which gcc
/alchemy-cygwin-v0.5a/achacks/gcc
$ gcc -v
Using built-in specs.
Target: i686-pc-cygwin
Configured with: /home/anon/llvm-gcc4.0-2.1.source/configure : (reconfigured) /
home/anon/llvm-gcc4.0-2.1.source/configure : (reconfigured) /home/anon/llvm-gcc
4.0-2.1.source/configure --disable-libgcj --disable-libjava : (reconfigured) ../
../src/llvm-gcc4/configure --prefix=/home/anon/llvm-gcc4 --disable-threads --dis
able-nls --disable-shared --enable-languages=c,c++ --disable-c-mbchar --program-
prefix=llvm- --enable-llvm=/home/anon/src/llvm : (reconfigured) /home/anon/src/l
lvm-gcc4/configure --prefix=/home/anon/llvm-gcc4 --disable-threads --disable-nls
--disable-shared --enable-languages=c,c++ --disable-c-mbchar --program-prefix=l
lvm- --enable-llvm=/home/anon/src/llvm
Thread model: single
gcc version 4.0.1 (Apple Computer, Inc. build 5449)
可以看到alchemy官方版附带的gcc版本是4,用苹果机编译。
二、编写代码编译运行,比较Alchemy和AS3实现的结果。
为了简单起见忽略huffman压缩和解压算法的实现代码。
Alchemy的API导出文件(.gg文件)
---------------------------
{ /* 注意用大括号包含C代码, 一些通常用到的C函数定义,方便C和AS3的类型转换 */ #include <stdlib.h> #include <string.h> #include <malloc.h> /* 需要用到的头文件 */ #include "Huffman.h" /* AS3.h is included automatically by gluegen */ void sztrace(char*); AS3_Val no_params = NULL; AS3_Val zero_param = NULL; AS3_Val ByteArray_class = NULL; AS3_Val getTimer_method = NULL; // 保存压缩大小 static int compress_size = 0; /* This function will be called at the top of the generated main(). The GGINIT_DEFINED macro is required. */ /* 必须预定义GGINIT_DEFINED */ #define GGINIT_DEFINED true static void ggInit() { //sztrace("setting up as3_crypto_wrapper library"); /* setup some useful constants */ no_params = AS3_Array(""); zero_param = AS3_Int(0); AS3_Val flash_utils_namespace = AS3_String("flash.utils"); ByteArray_class = AS3_NSGetS(flash_utils_namespace, "ByteArray"); getTimer_method = AS3_NSGetS(flash_utils_namespace, "getTimer"); AS3_Release(flash_utils_namespace); /* initialize */ /* 这里插入全局初始化代码*/ } /* Copy the byteArray data into a malloc'd buffer */ /* ByteArray转void* */ static void* newMallocFromByteArray(AS3_Val byteArray, unsigned int* size) { AS3_Val byteArraySize = AS3_GetS(byteArray, "length"); *size = AS3_IntValue(byteArraySize); AS3_Release(byteArraySize); void* bytes = malloc(*size); AS3_SetS(byteArray, "position", zero_param); AS3_ByteArray_readBytes((char*)bytes, byteArray, (int)*size); return bytes; } /* Make a new ByteArray containing the data passed */ /* void*转ByteArray */ static AS3_Val newByteArrayFromMalloc(void *data, unsigned int size) { AS3_Val byteArray = AS3_New(ByteArray_class, no_params); AS3_ByteArray_writeBytes(byteArray, data, size); return byteArray; } #if 0 // use for profiling /* get a timestamp from Flash */ static int getTimestamp() { AS3_Val ts = AS3_Call(getTimer_method, NULL, no_params); int result = AS3_IntValue(ts); AS3_Release(ts); return result; } #endif /* C代码结束 */ /* 下面就可以加入要在swc中暴露的API声明 */ } public function huff_uncompress(indata:ByteArray, insize:uint, outsize:uint):ByteArray { unsigned int size = 0; unsigned char* inbuf = newMallocFromByteArray(indata, &size); unsigned char* outbuf = malloc(outsize); //测试用 //memset(outbuf, outsize, 10); Huffman_Uncompress(inbuf, outbuf, insize, outsize); AS3_Val ba = newByteArrayFromMalloc(outbuf, outsize); free(inbuf); free(outbuf); return ba; } public function huff_compress(indata:ByteArray, insize:uint):ByteArray { unsigned int size = 0; unsigned char* inbuf = newMallocFromByteArray(indata, &size); //足够容纳压缩内容的大小 unsigned int outsize = (insize*104+50)/100 + 384; unsigned char* outbuf = malloc(outsize); //测试用 //memset(outbuf, outsize, 10); //调试用 /* char strTrace[1000] = ""; sprintf(strTrace, "size: %d, outsize:%d, insize:%d", size, outsize, insize); sztrace(strTrace); sztrace(inbuf); */ compress_size = Huffman_Compress(inbuf, outbuf, insize); AS3_Val ba = newByteArrayFromMalloc(outbuf, compress_size); free(inbuf); free(outbuf); return ba; } public function huff_compress_size():uint { // 取出最近压缩的大小 return compress_size; }
----------------------------------------------
Alchemy实现的主入口代码如下:
package { import flash.display.Sprite; import flash.events.Event; import flash.utils.ByteArray; import flash.utils.Endian; /** * ... * @author */ public class Main extends Sprite { import cmodule.huff.CLibInit; private static const clibinit:CLibInit = new CLibInit(); private static const hufflib:Object = clibinit.init(); public function Main():void { if (stage) init(); else addEventListener(Event.ADDED_TO_STAGE, init); } private function init(e:Event = null):void { removeEventListener(Event.ADDED_TO_STAGE, init); // entry point //-------------------------------------------------- [Embed(source='../lib/output.dat', mimeType='application/octet-stream')] var output_dat:Class; //注意,是强制转换,不是new var data:ByteArray = ByteArray(new output_dat()); trace("ByteArray.length = ", data.length); data.position = 0; //不知为何,与网络传输时正好相反 //0xFF000000 --readUnsignedInt--> 0xFF data.endian = Endian.LITTLE_ENDIAN; if (data.bytesAvailable > 0) { // 第一个32位数是解压长度 var uncompress_length:uint = data.readUnsignedInt(); var inputdata:ByteArray = new ByteArray(); data.readBytes(inputdata); trace("inputdata.bytesAvailable", inputdata.bytesAvailable); trace("uncompress_length:", uncompress_length); inputdata.position = 0; //解码 var uncompress_bytes:ByteArray = huff_uncompress(inputdata, inputdata.bytesAvailable, uncompress_length); uncompress_bytes.position = 0; trace("uncompress:", uncompress_bytes.bytesAvailable, "bytes"); var str:String = uncompress_bytes.readMultiByte(uncompress_bytes.bytesAvailable, "gbk"); trace(str); } //-------------------------------------------------- //压缩测试 /* var compress_length:uint = huff_compress_size(); trace("compress_length", compress_length); */ /**/ uncompress_bytes.position = 0; trace("uncompress_bytes.bytesAvailable:", uncompress_bytes.bytesAvailable); var compress_bytes:ByteArray = huff_compress(uncompress_bytes, uncompress_bytes.bytesAvailable); var compress_length:uint = huff_compress_size(); compress_bytes.position = 0; trace("compress_length:", compress_length); trace("compress:", compress_bytes.bytesAvailable, "bytes"); compress_bytes.position = 0; while (compress_bytes.bytesAvailable > 0) { var byte:uint = compress_bytes.readByte() & 0xff; trace(byte.toString(16)); } /**/ } //对应huff.gg中的三个导出API //方便参数类型检查 public function huff_uncompress(bytes:ByteArray, insize:int, outsize:int):ByteArray { return hufflib.huff_uncompress(bytes, insize, outsize); } public function huff_compress(bytes:ByteArray, insize:int):ByteArray { //trace("insize:", insize); return hufflib.huff_compress(bytes, insize); } public function huff_compress_size():int { return hufflib.huff_compress_size(); } } }
-------------------------------------------------
纯AS3实现的主入口代码如下:
package huff { import flash.display.Sprite; import flash.utils.ByteArray; import flash.utils.Endian; public class CodecTest extends Sprite { public function CodecTest() { /* trace("hello, world"); var bytes:ByteArray = new ByteArray(); bytes.writeByte(0xF8); bytes.writeByte(0xF8); bytes.position = 0; var stream:Bitstream = new Bitstream(bytes); try { //testReadBit(stream); testReadBit(stream); testReadBit(stream); testReadBit(stream); testReadBit(stream); testReadBit(stream); testReadBit(stream); testReadBit(stream); testRead8Bits(stream); testRead8Bits(stream); testRead8Bits(stream); }catch (e:Error) { trace(e.getStackTrace()); } //---------------------------------------------------- stream = new Bitstream(); stream.writeBits(0xF8, 8); stream.writeBits(0xF8, 8); stream.rewind(); try { //testReadBit(stream); testReadBit(stream); testReadBit(stream); testReadBit(stream); testReadBit(stream); testReadBit(stream); testReadBit(stream); testReadBit(stream); testRead8Bits(stream); testRead8Bits(stream); testRead8Bits(stream); }catch (e:Error) { trace(e.getStackTrace()); } */ var stream:Bitstream = new Bitstream(); //-------------------------------------------------- [Embed(source='../../lib/output.dat', mimeType='application/octet-stream')] var output_dat:Class; //注意,是强制转换,不是new var data:ByteArray = ByteArray(new output_dat()); trace("ByteArray.length = ", data.length); data.position = 0; //不知为何,与网络传输时正好相反 //0xFF000000 --readUnsignedInt--> 0xFF data.endian = Endian.LITTLE_ENDIAN; if (data.bytesAvailable > 0) { // 第一个32位数是解压长度 var uncompress_length:uint = data.readUnsignedInt(); trace("uncompress_length:", uncompress_length); stream = new Bitstream(data); var uncompress_bytes:ByteArray = new ByteArray(); stream.uncompress(uncompress_bytes, uncompress_length); uncompress_bytes.position = 0; trace("uncompress:", uncompress_bytes.bytesAvailable, "bytes"); var str:String = uncompress_bytes.readMultiByte(uncompress_bytes.bytesAvailable, "gbk"); trace(str); } //-------------------------------------------------- //压缩测试 uncompress_bytes.position = 0; var compress_bytes:ByteArray = new ByteArray(); var stream2:Bitstream = new Bitstream(); var compress_length:uint = stream2.compress(uncompress_bytes, compress_bytes, uncompress_bytes.bytesAvailable); compress_bytes.position = 0; trace("compress_length:", compress_length); trace("compress:", compress_bytes.bytesAvailable, "bytes"); compress_bytes.position = 0; while (compress_bytes.bytesAvailable > 0) { var byte:uint = compress_bytes.readByte() & 0xff; trace(byte.toString(16)); } } public function testRead8Bits(stream:Bitstream):void { var i:uint = stream.read8Bits(); trace(i.toString(16)); } public function testReadBit(stream:Bitstream):void { var i:uint = stream.readBit(); trace(i); } } }
--------------------------
两种实现的输出如下所示,
* 解压出"Hello World!",
* 压缩除了alchemy没有加入开头4字节的长度外,其余和AS3版相同。
---------------------
alchemy实现的输出:
inputdata.bytesAvailable 16
uncompress_length: 12
uncompress: 12 bytes
Hello World!
uncompress_bytes.bytesAvailable: 12
compress_length: 16
compress: 16 bytes
cd
f1
4
90
21
9b
33
21
5e
72
32
b7
d0
b4
eb
1c
------------------------
纯AS3实现的输出
uncompress_length: 12
uncompress: 12 bytes
Hello World!
compress_length: 16
compress: 20 bytes
c
0
0
0
cd
f1
4
90
21
9b
33
21
5e
72
32
b7
d0
b4
eb
1c
------------------------
结论:
Alchemy和纯AS3都可以处理ByteArray数据,进行类似数据压缩和解压的复杂工作。
从上面的实践可以发现Alchemy开发的一些注意事项:
* 开发环境比较难搭建,需要耐心寻找原因。
* 难于调试。需要使用全局函数sztrace跟踪代码的去向。
* 如果遗漏了.o文件仍可以编译swc成功,但会导致运行期错误,所以写Makefile时必须十分小心。
* .gg文件必须把公共的C代码放在开头的大括号内。导出API放在大括号外。
代码在svn中:
svn://www.svnhost.cn/weimingtom_reversi/alchemy