Snappy压缩库安装和使用之一

最新推荐文章于 2024-07-02 20:47:04 发布
最新推荐文章于 2024-07-02 20:47:04 发布
阅读量1.4k 收藏 1
点赞数
文章标签： 开发工具 shell 运维

近日需要在毕业设计中引入一个压缩库，要求压缩与解压缩速度快，但是压缩率可以不那么苛刻。查找资料发现Google的snappy库比较合适，而且该库开源，由C++写成。所以就拿来使用一下，下面权作记录。下面引出的任何涉及Google公司的源代码，版权归Google公司所有，我权作学习交流。文章安排如下，首先简要介绍Snappy，之后安装之，然后以实际例子介绍如何使用，接着bzip2和gzip做了性能比较，最后提出一些使用上面的疑问。

（一）简要介绍

去官网下载之http://code.google.com/p/snappy/。在ProjectHome处有这么一段英文，我想许多地方都引用和翻译了这段。我也尝试翻译一下。

Snappyisacompression/decompressionlibrary.

Itdoesnotaimformaximumcompression,

orcompatibilitywithanyothercompressionlibrary;

instead,itaimsforveryhighspeedsandreasonablecompression.

Forinstance,comparedtothefastestmodeofzlib,

Snappyisanorderofmagnitudefasterformostinputs,

buttheresultingcompressedfilesareanywherefrom20%to100%bigger.

OnasinglecoreofaCorei7processorin64-bitmode,

Snappycompressesatabout250MB/secormoreand

decompressesatabout500MB/secormore.

SnappyiswidelyusedinsideGoogle,ineverythingfromBigTable

andMapReducetoourinternalRPCsystems.

译文：Snappy是一个压缩/解压缩库。它不是以最大压缩率，或者与其他压缩库兼容为目标；它旨在获得高速的压缩和合理的压缩率。例如，Snappy对大多数的输入比zlib的最快模式要快几个数量级，但是其压缩过后的文件通常会比zlib大20%到100%。在Corei7的单核64位模式下，Snappy压缩速度大概可以达到250MB/s或者更快，解压缩可以达到大约500MB/s或更快。

Snappy在Google内部广泛使用，从BigTable，MapReduce到公司内部的RPC系统。

（二）安装过程

下面描述安装过程：

下载snappy-1.0.5.tar.gz，snappy的安装过程与传统的安装过程一样。解压后的INSTALL文件有详细的安装说明。

gunzipsnappy-1.0.5.tar.gz

tarxfsnappy-1.0.5.tar

cdsnappy-1.0.5

./configure

make

makeinstall

安装完成后，生成的动态库和静态库位于/usr/local/lib处，编程需要用到的头文件位于/usr/local/include处。注意需要将这些库文件cp至/usr/lib处，不然就算在链接的时候加上-L/usr/local/lib，在运行时也会报错。./main:errorwhileloadingsharedlibraries:libsnappy.so.1:

cannotopensharedobjectfile:Nosuchfileordirectory

当然这是我的LD_LIBRARY_PATH环境变量的设置问题。

（三）使用snappy

解压出来的README文件介绍了一简单的使用方式。snappy是各种库标示符所在的命名空间。C++使用需要包含#include<snappy.h>头文件，C语言使用需要包含#include<snapyy-c.h>头文件。Snappy使用较为简单，我指的是跟bzip2的库比起来。所有的函数接口都暴露在上述两个头文件中，头文件中有详细的使用说明，并有简单的示例，而且英文通俗易懂。摘抄如下（Google公司版权所有）：

snappy.h

[cpp] view plain copy print ?
  1. //Copyright2005andonwardsGoogleInc.
  2. //
  3. //Redistributionanduseinsourceandbinaryforms,withorwithout
  4. //modification,arepermittedprovidedthatthefollowingconditionsare
  5. //met:
  6. //
  7. //*Redistributionsofsourcecodemustretaintheabovecopyright
  8. //notice,thislistofconditionsandthefollowingdisclaimer.
  9. //*Redistributionsinbinaryformmustreproducetheabove
  10. //copyrightnotice,thislistofconditionsandthefollowingdisclaimer
  11. //inthedocumentationand/orothermaterialsprovidedwiththe
  12. //distribution.
  13. //*NeitherthenameofGoogleInc.northenamesofits
  14. //contributorsmaybeusedtoendorseorpromoteproductsderivedfrom
  15. //thissoftwarewithoutspecificpriorwrittenpermission.
  16. //
  17. //THISSOFTWAREISPROVIDEDBYTHECOPYRIGHTHOLDERSANDCONTRIBUTORS
  18. //"ASIS"ANDANYEXPRESSORIMPLIEDWARRANTIES,INCLUDING,BUTNOT
  19. //LIMITEDTO,THEIMPLIEDWARRANTIESOFMERCHANTABILITYANDFITNESSFOR
  20. //APARTICULARPURPOSEAREDISCLAIMED.INNOEVENTSHALLTHECOPYRIGHT
  21. //OWNERORCONTRIBUTORSBELIABLEFORANYDIRECT,INDIRECT,INCIDENTAL,
  22. //SPECIAL,EXEMPLARY,ORCONSEQUENTIALDAMAGES(INCLUDING,BUTNOT
  23. //LIMITEDTO,PROCUREMENTOFSUBSTITUTEGOODSORSERVICES;LOSSOFUSE,
  24. //DATA,ORPROFITS;ORBUSINESSINTERRUPTION)HOWEVERCAUSEDANDONANY
  25. //THEORYOFLIABILITY,WHETHERINCONTRACT,STRICTLIABILITY,ORTORT
  26. //(INCLUDINGNEGLIGENCEOROTHERWISE)ARISINGINANYWAYOUTOFTHEUSE
  27. //OFTHISSOFTWARE,EVENIFADVISEDOFTHEPOSSIBILITYOFSUCHDAMAGE.
  28. //
  29. //Alight-weightcompressionalgorithm.Itisdesignedforspeedof
  30. //compressionanddecompression,ratherthanfortheutmostinspace
  31. //savings.
  32. //
  33. //Forgettingbettercompressionratioswhenyouarecompressingdata
  34. //withlongrepeatedsequencesorcompressingdatathatissimilarto
  35. //otherdata,whilestillcompressingfast,youmightlookatfirst
  36. //usingBMDiffandthencompressingtheoutputofBMDiffwith
  37. //Snappy.
  39. #ifndefUTIL_SNAPPY_SNAPPY_H__
  40. #defineUTIL_SNAPPY_SNAPPY_H__
  42. #include<stddef.h>
  43. #include<string>
  45. #include"snappy-stubs-public.h"
  47. namespacesnappy{
  48. classSource;
  49. classSink;
  51. //------------------------------------------------------------------------
  52. //Genericcompression/decompressionroutines.
  53. //------------------------------------------------------------------------
  55. //Compressthebytesreadfrom"*source"andappendto"*sink".Returnthe
  56. //numberofbyteswritten.
  57. size_tCompress(Source*source,Sink*sink);
  59. boolGetUncompressedLength(Source*source,uint32*result);
  61. //------------------------------------------------------------------------
  62. //Higher-levelstringbasedroutines(shouldbesufficientformostusers)
  63. //------------------------------------------------------------------------
  65. //Sets"*output"tothecompressedversionof"input[0,input_length-1]".
  66. //Originalcontentsof*outputarelost.
  67. //
  68. //REQUIRES:"input[]"isnotanaliasof"*output".
  69. size_tCompress(constchar*input,size_tinput_length,string*output);
  71. //Decompresses"compressed[0,compressed_length-1]"to"*uncompressed".
  72. //Originalcontentsof"*uncompressed"arelost.
  73. //
  74. //REQUIRES:"compressed[]"isnotanaliasof"*uncompressed".
  75. //
  76. //returnsfalseifthemessageiscorruptedandcouldnotbedecompressed
  77. boolUncompress(constchar*compressed,size_tcompressed_length,
  78. string*uncompressed);
  81. //------------------------------------------------------------------------
  82. //Lower-levelcharacterarraybasedroutines.Maybeusefulfor
  83. //efficiencyreasonsincertaincircumstances.
  84. //------------------------------------------------------------------------
  86. //REQUIRES:"compressed"mustpointtoanareaofmemorythatisat
  87. //least"MaxCompressedLength(input_length)"bytesinlength.
  88. //
  89. //Takesthedatastoredin"input[0..input_length]"andstores
  90. //itinthearraypointedtoby"compressed".
  91. //
  92. //"*compressed_length"issettothelengthofthecompressedoutput.
  93. //
  94. //Example:
  95. //char*output=newchar[snappy::MaxCompressedLength(input_length)];
  96. //size_toutput_length;
  97. //RawCompress(input,input_length,output,&output_length);
  98. //...Process(output,output_length)...
  99. //delete[]output;
  100. voidRawCompress(constchar*input,
  101. size_tinput_length,
  102. char*compressed,
  103. size_t*compressed_length);
  105. //Givendatain"compressed[0..compressed_length-1]"generatedby
  106. //callingtheSnappy::Compressroutine,thisroutine
  107. //storestheuncompresseddatato
  108. //uncompressed[0..GetUncompressedLength(compressed)-1]
  109. //returnsfalseifthemessageiscorruptedandcouldnotbedecrypted
  110. boolRawUncompress(constchar*compressed,size_tcompressed_length,
  111. char*uncompressed);
  113. //Givendatafromthebytesource'compressed'generatedbycalling
  114. //theSnappy::Compressroutine,thisroutinestorestheuncompressed
  115. //datato
  116. //uncompressed[0..GetUncompressedLength(compressed,compressed_length)-1]
  117. //returnsfalseifthemessageiscorruptedandcouldnotbedecrypted
  118. boolRawUncompress(Source*compressed,char*uncompressed);
  120. //Returnsthemaximalsizeofthecompressedrepresentationof
  121. //inputdatathatis"source_bytes"bytesinlength;
  122. size_tMaxCompressedLength(size_tsource_bytes);
  124. //REQUIRES:"compressed[]"wasproducedbyRawCompress()orCompress()
  125. //Returnstrueandstoresthelengthoftheuncompresseddatain
  126. //*resultnormally.Returnsfalseonparsingerror.
  127. //ThisoperationtakesO(1)time.
  128. boolGetUncompressedLength(constchar*compressed,size_tcompressed_length,
  129. size_t*result);
  131. //Returnstrueiffthecontentsof"compressed[]"canbeuncompressed
  132. //successfully.Doesnotreturntheuncompresseddata.Takes
  133. //timeproportionaltocompressed_length,butisusuallyatleast
  134. //afactoroffourfasterthanactualdecompression.
  135. boolIsValidCompressedBuffer(constchar*compressed,
  136. size_tcompressed_length);
  138. //***DONOTCHANGETHEVALUEOFkBlockSize***
  139. //
  140. //NewCompressioncodechopsuptheinputintoblocksofatmost
  141. //thefollowingsize.Thisensuresthatback-referencesinthe
  142. //outputnevercrosskBlockSizeblockboundaries.Thiscanbe
  143. //helpfulinimplementingblockeddecompression.Howeverthe
  144. //decompressioncodeshouldnotrelyonthisguaranteesinceolder
  145. //compressioncodemaynotobeyit.
  146. staticconstintkBlockLog=15;
  147. staticconstsize_tkBlockSize=1<<kBlockLog;
  149. staticconstintkMaxHashTableBits=14;
  150. staticconstsize_tkMaxHashTableSize=1<<kMaxHashTableBits;
  152. }//endnamespacesnappy
  155. #endif//UTIL_SNAPPY_SNAPPY_H__
// Copyright 2005 and onwards Google Inc.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
//     * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
//     * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
//     * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
// A light-weight compression algorithm.  It is designed for speed of
// compression and decompression, rather than for the utmost in space
// savings.
//
// For getting better compression ratios when you are compressing data
// with long repeated sequences or compressing data that is similar to
// other data, while still compressing fast, you might look at first
// using BMDiff and then compressing the output of BMDiff with
// Snappy.

#ifndef UTIL_SNAPPY_SNAPPY_H__
#define UTIL_SNAPPY_SNAPPY_H__

#include <stddef.h>
#include <string>

#include "snappy-stubs-public.h"

namespace snappy {
  class Source;
  class Sink;

  // ------------------------------------------------------------------------
  // Generic compression/decompression routines.
  // ------------------------------------------------------------------------

  // Compress the bytes read from "*source" and append to "*sink". Return the
  // number of bytes written.
  size_t Compress(Source* source, Sink* sink);

  bool GetUncompressedLength(Source* source, uint32* result);

  // ------------------------------------------------------------------------
  // Higher-level string based routines (should be sufficient for most users)
  // ------------------------------------------------------------------------

  // Sets "*output" to the compressed version of "input[0,input_length-1]".
  // Original contents of *output are lost.
  //
  // REQUIRES: "input[]" is not an alias of "*output".
  size_t Compress(const char* input, size_t input_length, string* output);

  // Decompresses "compressed[0,compressed_length-1]" to "*uncompressed".
  // Original contents of "*uncompressed" are lost.
  //
  // REQUIRES: "compressed[]" is not an alias of "*uncompressed".
  //
  // returns false if the message is corrupted and could not be decompressed
  bool Uncompress(const char* compressed, size_t compressed_length,
                  string* uncompressed);


  // ------------------------------------------------------------------------
  // Lower-level character array based routines.  May be useful for
  // efficiency reasons in certain circumstances.
  // ------------------------------------------------------------------------

  // REQUIRES: "compressed" must point to an area of memory that is at
  // least "MaxCompressedLength(input_length)" bytes in length.
  //
  // Takes the data stored in "input[0..input_length]" and stores
  // it in the array pointed to by "compressed".
  //
  // "*compressed_length" is set to the length of the compressed output.
  //
  // Example:
  //    char* output = new char[snappy::MaxCompressedLength(input_length)];
  //    size_t output_length;
  //    RawCompress(input, input_length, output, &output_length);
  //    ... Process(output, output_length) ...
  //    delete [] output;
  void RawCompress(const char* input,
                   size_t input_length,
                   char* compressed,
                   size_t* compressed_length);

  // Given data in "compressed[0..compressed_length-1]" generated by
  // calling the Snappy::Compress routine, this routine
  // stores the uncompressed data to
  //    uncompressed[0..GetUncompressedLength(compressed)-1]
  // returns false if the message is corrupted and could not be decrypted
  bool RawUncompress(const char* compressed, size_t compressed_length,
                     char* uncompressed);

  // Given data from the byte source 'compressed' generated by calling
  // the Snappy::Compress routine, this routine stores the uncompressed
  // data to
  //    uncompressed[0..GetUncompressedLength(compressed,compressed_length)-1]
  // returns false if the message is corrupted and could not be decrypted
  bool RawUncompress(Source* compressed, char* uncompressed);

  // Returns the maximal size of the compressed representation of
  // input data that is "source_bytes" bytes in length;
  size_t MaxCompressedLength(size_t source_bytes);

  // REQUIRES: "compressed[]" was produced by RawCompress() or Compress()
  // Returns true and stores the length of the uncompressed data in
  // *result normally.  Returns false on parsing error.
  // This operation takes O(1) time.
  bool GetUncompressedLength(const char* compressed, size_t compressed_length,
                             size_t* result);

  // Returns true iff the contents of "compressed[]" can be uncompressed
  // successfully.  Does not return the uncompressed data.  Takes
  // time proportional to compressed_length, but is usually at least
  // a factor of four faster than actual decompression.
  bool IsValidCompressedBuffer(const char* compressed,
                               size_t compressed_length);

  // *** DO NOT CHANGE THE VALUE OF kBlockSize ***
  //
  // New Compression code chops up the input into blocks of at most
  // the following size.  This ensures that back-references in the
  // output never cross kBlockSize block boundaries.  This can be
  // helpful in implementing blocked decompression.  However the
  // decompression code should not rely on this guarantee since older
  // compression code may not obey it.
  static const int kBlockLog = 15;
  static const size_t kBlockSize = 1 << kBlockLog;

  static const int kMaxHashTableBits = 14;
  static const size_t kMaxHashTableSize = 1 << kMaxHashTableBits;

}  // end namespace snappy


#endif  // UTIL_SNAPPY_SNAPPY_H__

snapp-c.h

[cpp] view plain copy print ?
  1. /*
  2. *Copyright2011MartinGieseking<martin.gieseking@uos.de>.
  3. *
  4. *Redistributionanduseinsourceandbinaryforms,withorwithout
  5. *modification,arepermittedprovidedthatthefollowingconditionsare
  6. *met:
  7. *
  8. **Redistributionsofsourcecodemustretaintheabovecopyright
  9. *notice,thislistofconditionsandthefollowingdisclaimer.
  10. **Redistributionsinbinaryformmustreproducetheabove
  11. *copyrightnotice,thislistofconditionsandthefollowingdisclaimer
  12. *inthedocumentationand/orothermaterialsprovidedwiththe
  13. *distribution.
  14. **NeitherthenameofGoogleInc.northenamesofits
  15. *contributorsmaybeusedtoendorseorpromoteproductsderivedfrom
  16. *thissoftwarewithoutspecificpriorwrittenpermission.
  17. *
  18. *THISSOFTWAREISPROVIDEDBYTHECOPYRIGHTHOLDERSANDCONTRIBUTORS
  19. *"ASIS"ANDANYEXPRESSORIMPLIEDWARRANTIES,INCLUDING,BUTNOT
  20. *LIMITEDTO,THEIMPLIEDWARRANTIESOFMERCHANTABILITYANDFITNESSFOR
  21. *APARTICULARPURPOSEAREDISCLAIMED.INNOEVENTSHALLTHECOPYRIGHT
  22. *OWNERORCONTRIBUTORSBELIABLEFORANYDIRECT,INDIRECT,INCIDENTAL,
  23. *SPECIAL,EXEMPLARY,ORCONSEQUENTIALDAMAGES(INCLUDING,BUTNOT
  24. *LIMITEDTO,PROCUREMENTOFSUBSTITUTEGOODSORSERVICES;LOSSOFUSE,
  25. *DATA,ORPROFITS;ORBUSINESSINTERRUPTION)HOWEVERCAUSEDANDONANY
  26. *THEORYOFLIABILITY,WHETHERINCONTRACT,STRICTLIABILITY,ORTORT
  27. *(INCLUDINGNEGLIGENCEOROTHERWISE)ARISINGINANYWAYOUTOFTHEUSE
  28. *OFTHISSOFTWARE,EVENIFADVISEDOFTHEPOSSIBILITYOFSUCHDAMAGE.
  29. *
  30. *PlainCinterface(awrapperaroundtheC++implementation).
  31. */
  33. #ifndefUTIL_SNAPPY_OPENSOURCE_SNAPPY_C_H_
  34. #defineUTIL_SNAPPY_OPENSOURCE_SNAPPY_C_H_
  36. #ifdef__cplusplus
  37. extern"C"{
  38. #endif
  40. #include<stddef.h>
  42. /*
  43. *Returnvalues;seethedocumentationforeachfunctiontoknow
  44. *whateachcanreturn.
  45. */
  46. typedefenum{
  47. SNAPPY_OK=0,
  48. SNAPPY_INVALID_INPUT=1,
  49. SNAPPY_BUFFER_TOO_SMALL=2,
  50. }snappy_status;
  52. /*
  53. *Takesthedatastoredin"input[0..input_length-1]"andstores
  54. *itinthearraypointedtoby"compressed".
  55. *
  56. *<compressed_length>signalsthespaceavailablein"compressed".
  57. *Ifitisnotatleastequalto"snappy_max_compressed_length(input_length)",
  58. *SNAPPY_BUFFER_TOO_SMALLisreturned.Aftersuccessfulcompression,
  59. *<compressed_length>containsthetruelengthofthecompressedoutput,
  60. *andSNAPPY_OKisreturned.
  61. *
  62. *Example:
  63. *size_toutput_length=snappy_max_compressed_length(input_length);
  64. *char*output=(char*)malloc(output_length);
  65. *if(snappy_compress(input,input_length,output,&output_length)
  66. *==SNAPPY_OK){
  67. *...Process(output,output_length)...
  68. *}
  69. *free(output);
  70. */
  71. snappy_statussnappy_compress(constchar*input,
  72. size_tinput_length,
  73. char*compressed,
  74. size_t*compressed_length);
  76. /*
  77. *Givendatain"compressed[0..compressed_length-1]"generatedby
  78. *callingthesnappy_compressroutine,thisroutinestores
  79. *theuncompresseddatato
  80. *uncompressed[0..uncompressed_length-1].
  81. *Returnsfailure(avaluenotequaltoSNAPPY_OK)ifthemessage
  82. *iscorruptedandcouldnotbedecrypted.
  83. *
  84. *<uncompressed_length>signalsthespaceavailablein"uncompressed".
  85. *Ifitisnotatleastequaltothevaluereturnedby
  86. *snappy_uncompressed_lengthforthisstream,SNAPPY_BUFFER_TOO_SMALL
  87. *isreturned.Aftersuccessfuldecompression,<uncompressed_length>
  88. *containsthetruelengthofthedecompressedoutput.
  89. *
  90. *Example:
  91. *size_toutput_length;
  92. *if(snappy_uncompressed_length(input,input_length,&output_length)
  93. *!=SNAPPY_OK){
  94. *...fail...
  95. *}
  96. *char*output=(char*)malloc(output_length);
  97. *if(snappy_uncompress(input,input_length,output,&output_length)
  98. *==SNAPPY_OK){
  99. *...Process(output,output_length)...
  100. *}
  101. *free(output);
  102. */
  103. snappy_statussnappy_uncompress(constchar*compressed,
  104. size_tcompressed_length,
  105. char*uncompressed,
  106. size_t*uncompressed_length);
  108. /*
  109. *Returnsthemaximalsizeofthecompressedrepresentationof
  110. *inputdatathatis"source_length"bytesinlength.
  111. */
  112. size_tsnappy_max_compressed_length(size_tsource_length);
  114. /*
  115. *REQUIRES:"compressed[]"wasproducedbysnappy_compress()
  116. *ReturnsSNAPPY_OKandstoresthelengthoftheuncompresseddatain
  117. **resultnormally.ReturnsSNAPPY_INVALID_INPUTonparsingerror.
  118. *ThisoperationtakesO(1)time.
  119. */
  120. snappy_statussnappy_uncompressed_length(constchar*compressed,
  121. size_tcompressed_length,
  122. size_t*result);
  124. /*
  125. *Checkifthecontentsof"compressed[]"canbeuncompressedsuccessfully.
  126. *Doesnotreturntheuncompresseddata;ifso,returnsSNAPPY_OK,
  127. *orifnot,returnsSNAPPY_INVALID_INPUT.
  128. *Takestimeproportionaltocompressed_length,butisusuallyatleasta
  129. *factoroffourfasterthanactualdecompression.
  130. */
  131. snappy_statussnappy_validate_compressed_buffer(constchar*compressed,
  132. size_tcompressed_length);
  134. #ifdef__cplusplus
  135. }//extern"C"
  136. #endif
  138. #endif/*UTIL_SNAPPY_OPENSOURCE_SNAPPY_C_H_*/
/*
 * Copyright 2011 Martin Gieseking <martin.gieseking@uos.de>.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions are
 * met:
 *
 *     * Redistributions of source code must retain the above copyright
 * notice, this list of conditions and the following disclaimer.
 *     * Redistributions in binary form must reproduce the above
 * copyright notice, this list of conditions and the following disclaimer
 * in the documentation and/or other materials provided with the
 * distribution.
 *     * Neither the name of Google Inc. nor the names of its
 * contributors may be used to endorse or promote products derived from
 * this software without specific prior written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
 * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
 * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
 * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
 * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
 * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
 * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
 * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
 * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 *
 * Plain C interface (a wrapper around the C++ implementation).
 */

#ifndef UTIL_SNAPPY_OPENSOURCE_SNAPPY_C_H_
#define UTIL_SNAPPY_OPENSOURCE_SNAPPY_C_H_

#ifdef __cplusplus
extern "C" {
#endif

#include <stddef.h>

/*
 * Return values; see the documentation for each function to know
 * what each can return.
 */
typedef enum {
  SNAPPY_OK = 0,
  SNAPPY_INVALID_INPUT = 1,
  SNAPPY_BUFFER_TOO_SMALL = 2,
} snappy_status;

/*
 * Takes the data stored in "input[0..input_length-1]" and stores
 * it in the array pointed to by "compressed".
 *
 * <compressed_length> signals the space available in "compressed".
 * If it is not at least equal to "snappy_max_compressed_length(input_length)",
 * SNAPPY_BUFFER_TOO_SMALL is returned. After successful compression,
 * <compressed_length> contains the true length of the compressed output,
 * and SNAPPY_OK is returned.
 *
 * Example:
 *   size_t output_length = snappy_max_compressed_length(input_length);
 *   char* output = (char*)malloc(output_length);
 *   if (snappy_compress(input, input_length, output, &output_length)
 *       == SNAPPY_OK) {
 *     ... Process(output, output_length) ...
 *   }
 *   free(output);
 */
snappy_status snappy_compress(const char* input,
                              size_t input_length,
                              char* compressed,
                              size_t* compressed_length);

/*
 * Given data in "compressed[0..compressed_length-1]" generated by
 * calling the snappy_compress routine, this routine stores
 * the uncompressed data to
 *   uncompressed[0..uncompressed_length-1].
 * Returns failure (a value not equal to SNAPPY_OK) if the message
 * is corrupted and could not be decrypted.
 *
 * <uncompressed_length> signals the space available in "uncompressed".
 * If it is not at least equal to the value returned by
 * snappy_uncompressed_length for this stream, SNAPPY_BUFFER_TOO_SMALL
 * is returned. After successful decompression, <uncompressed_length>
 * contains the true length of the decompressed output.
 *
 * Example:
 *   size_t output_length;
 *   if (snappy_uncompressed_length(input, input_length, &output_length)
 *       != SNAPPY_OK) {
 *     ... fail ...
 *   }
 *   char* output = (char*)malloc(output_length);
 *   if (snappy_uncompress(input, input_length, output, &output_length)
 *       == SNAPPY_OK) {
 *     ... Process(output, output_length) ...
 *   }
 *   free(output);
 */
snappy_status snappy_uncompress(const char* compressed,
                                size_t compressed_length,
                                char* uncompressed,
                                size_t* uncompressed_length);

/*
 * Returns the maximal size of the compressed representation of
 * input data that is "source_length" bytes in length.
 */
size_t snappy_max_compressed_length(size_t source_length);

/*
 * REQUIRES: "compressed[]" was produced by snappy_compress()
 * Returns SNAPPY_OK and stores the length of the uncompressed data in
 * *result normally. Returns SNAPPY_INVALID_INPUT on parsing error.
 * This operation takes O(1) time.
 */
snappy_status snappy_uncompressed_length(const char* compressed,
                                         size_t compressed_length,
                                         size_t* result);

/*
 * Check if the contents of "compressed[]" can be uncompressed successfully.
 * Does not return the uncompressed data; if so, returns SNAPPY_OK,
 * or if not, returns SNAPPY_INVALID_INPUT.
 * Takes time proportional to compressed_length, but is usually at least a
 * factor of four faster than actual decompression.
 */
snappy_status snappy_validate_compressed_buffer(const char* compressed,
                                                size_t compressed_length);

#ifdef __cplusplus
}  // extern "C"
#endif

#endif  /* UTIL_SNAPPY_OPENSOURCE_SNAPPY_C_H_ */


承影v
关注
  • 0
    点赞
  • 踩
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
golang中snappy的使用场合实例详解
09-20
在java 和go语言 大字符传达的时候， 采用snappy 压缩 解压缩是最好的方案。下面这篇文章主要给大家介绍了关于golang中snappy使用场合的相关资料，文中通过示例代码介绍的非常详细，需要的朋友可以参考下。
hadoop2.7.2 之 snappy压缩支持包.zip
05-27
替换完成后，Hadoop在运行时会自动识别并使用新的Snappy压缩库，从而实现对Snappy压缩格式的支持。 在实际应用中，可以通过修改Hadoop配置文件（如`core-site.xml`）来指定默认的压缩算法为Snappy。例如，可以添加...
Snappy使用
最新发布
weixin_45010335的博客
07-02 404
Snappy是谷歌开源的压缩和解压的开发包，目标在于实现高速的压缩而不是最大的压缩。
使用Snappy将html或者url转成PDF文件
weixin_30267785的博客
12-12 467
这是一个操作简单的html文件或者url转PDF的php库 Github地址https://github.com/KnpLabs/snappy 安装： $ composer require knplabs/knp-snappy $ composer require h4cc/wkhtmltopdf-amd64 0.12.x $ composer require h4cc/wkh...
.NET下使用snappy
whu_xxie的专栏
06-18 744
Snappy 是一个 C++ 的用来压缩和解压缩的开发包。其目标不是最大限度压缩或者兼容其他压缩格式，而是旨在提供高速压缩速度和合理的压缩率。Snappy 比 zlib 更快，但文件相对要大 20% 到 100%。在 64位模式的 Core i7 处理器上，可达每秒 250~500兆的压缩速度。 Snappy 在 Google 内部被广泛的使用，从 BigTable 到 MapReduce 以及内部的 RPC 系统。 snappy 是一个压缩库。它可以利用单颗Intel Corei7处理器内核处理至少每
11Snappy-压缩工具-windows
10-12
总的来说，安装和配置Snappy压缩工具对于在Windows环境中搭建Caffe深度学习框架是必要的步骤之一，它有助于提高数据处理速度，优化内存占用，使得Caffe能够更高效地运行大规模的机器学习任务。正确安装和设置这些...
c#压缩算法源码库
05-26
最后，zlib是最广泛使用的压缩库之一，它的算法基于DEFLATE，是Gzip和ZIP格式的基础。Zlib不仅提供了压缩和解压缩功能，还包含了错误检测和处理机制。在C#中，.NET框架本身就已经内置了对zlib的支持，通过System.IO....
Hadoop HBase 配置 安装 Snappy 终极教程
05-03
Snappy则是一种高效的压缩和解压缩库，常用于提升大数据系统中的I/O性能。本文将深入探讨如何配置和安装Hadoop HBase以及集成Snappy，以优化大数据处理的效率。 首先，我们需要理解Hadoop的环境配置。Hadoop的配置...
hadoop-snappy-0.0.1-SNAPSHOT.tar.gz
11-09
一、Snappy压缩库介绍 Snappy是由Google开发的一种快速、轻量级的压缩和解压缩库，主要设计目标是提供高吞吐量而不是最小化的压缩比。它在处理大量数据时表现优秀，广泛应用于大数据环境，如Hadoop和HBase。Snappy...
snappy压缩，解压算法
07-07
google snappy 快速压缩算法，c语言版本 速度比zlib快10倍，内存占用大概多25%
snappy压缩过程详解
03-25
对snappy压缩研究后的算法详解，有兴趣的可以看看
C++资源大全之压缩、并发性、容器
08-29
* bzip2：一个完全免费、免费专利和高质量的数据压缩库，能够对数据进行高效的压缩和解压缩。 * doboz：能够快速解压缩的压缩库，适用于需要快速解压缩的应用场景。 * PhysicsFS：对各种归档提供抽象访问的库，主要...
通过Java测试几种压缩算法的性能（附测试代码下载）
09-02
Snappy是Google开发的一种压缩算法，旨在兼顾速度和压缩比。它在许多Google项目中被广泛使用，因为它能够在保持相对良好的压缩效果的同时，提供快速的压缩和解压缩速度。 为了进行性能测试，作者选择了一个常见的、...
hadoop-snappy的jar包
03-15
Snappy是一种由Google开发的高效、快速的压缩和解压缩算法，它在Hadoop中被用作数据压缩的选项之一。然而，对于Hadoop用户来说，直接使用Snappy功能需要对应的Java Archive (JAR) 文件，即hadoop-snappy的jar包。 ...
「Snappy」- 使用 snap 安装应用 @20210208
k4nz
02-08 326
安装应用 我们以安装 kubectl 命令为例： 第一步、先进行应用的搜索 # snap search kubectl Name Version Publisher Notes Summary kubectl 1.18.8 canonical✓ classic Command line client for cont.
[转]Snappy压缩库安装和使用之一
weixin_30677073的博客
01-06 285
Snappy压缩库安装和使用之一 原文地址：http://blog.csdn.net/luo6620378xu/article/details/8521223 近日需要在毕业设计中引入一个压缩库，要求压缩与解压缩速度快，但是压缩率可以不那么苛刻。查找资料发现Google的snappy库比较合适，而且该库开源，由C++写成。所以就拿来使用一下，下面权作记录。下面引出的...
snappy压缩
cuidongdong1234的专栏
11-11 2108
Snappy 是一个 C++ 的用来压缩和解压缩的开发包，其目标不是最大限度压缩，而且不兼容其他压缩格式。Snappy 旨在提供高速压缩速度和合理的压缩率。Snappy 比 zlib 更快，但文件相对要大 20% 到 100%。在 64位模式的 Core i7 处理器上，可达每秒 250~500兆的压缩速度。      Snappy 在 Google 内部被广泛的使用，从 BigTable 到
windows安装snappy库
03-16
在Windows上安装Snappy库可以按照以下步骤进行操作： 1. 首先，确保你已经安装了CMake和Visual Studio。如果没有安装，你可以从官方网站下载并安装它们。 2. 下载Snappy的源代码。你可以从GitHub上找到Snappy的源代码仓库。 3. 解压下载的源代码文件，并进入解压后的文件夹。 4. 在解压后的文件夹中创建一个名为"build"的文件夹，用于存放编译生成的文件。 5. 打开命令提示符或PowerShell，并进入到"build"文件夹所在的路径。 6. 在命令提示符或PowerShell中运行以下命令来生成Visual Studio项目文件： ``` cmake .. ``` 7. 执行完上述命令后，会生成一个Visual Studio的解决方案文件（.sln）。双击打开该解决方案文件。 8. 在Visual Studio中，选择合适的配置（Debug或Release）和平台（x86或x64）。 9. 编译项目，在Visual Studio的菜单栏中选择"Build" -> "Build Solution"。 10. 编译完成后，在"build"文件夹中会生成相应的库文件（.lib）和头文件（.h）。 11. 将生成的库文件和头文件拷贝到你的项目中，并在代码中引用Snappy库即可使用。 希望以上步骤对你有帮助！
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值