.net(c#)读取*.dbf文件数据的类库

新制作了一个库存管理软件(c/s),数据库用mssql,客户端用.net开发。

用户在客户端对数据进行查看、添加、导入、编辑、删除、打印等操作。

公司原来的一些数据存储在.dbf文件中,必须在客户端对.dbf文件进行访问,以导入数据。

从网上搜集到关于.dbf文件格式的信息和一部分c语言直接访问.dbf文件的实例,写了一个c#直接访问.dbf的类,支持直接获取.dbf文件的DataSet,获取单个字段信息等。

1。定义.dbf文件头类


namespace  Sunman.DBF
{
    
using System;

    
/// <summary>
    
/// .dbf 文件的文件头信息类
    
/// </summary>

    internal class DBFHeader
    
{
        
public const int DBFHeaderSize = 32;
        
/* 版本标志
                     0x02    FoxBASE  
                    0x03    FoxBASE+/dBASE III PLUS,无备注  
                    0x30    Visual FoxPro  
                    0x43    dBASE IV SQL 表文件,无备注  
                    0x63    dBASE IV SQL 系统文件,无备注  
                    0x83    FoxBASE+/dBASE III PLUS,有备注  
                    0x8B    dBASE IV 有备注  
                    0xCB    dBASE IV SQL 表文件,有备注  
                    0xF5    FoxPro 2.x(或更早版本)有备注  
                    0xFB    FoxBASE  
        
*/

        
public sbyte Version;
        
/* 最后更新年 */ 
        
public byte LastModifyYear;
        
/* 最后更新月 */
        
public byte LastModifyMonth;
        
/* 最后更新日 */
        
public byte LastModifyDay;
        
/* 文件包含的总记录数 */ 
        
public uint RecordCount;
        
/* 第一条记录的偏移值,这个值也可以表示文件头长度 */ 
        
public ushort HeaderLength;
        
/* 记录长度,包括删除标志*/
        
public ushort RecordLength;
        
/* 保留 */ 
        
public byte[] Reserved = new byte[16];
        
/* 表的标志
                     0x01具有 .cdx 结构的文件
                    0x02文件包含备注
                    0x04文件是数据库(.dbc) 
                    标志可OR 
        
*/

        
public sbyte TableFlag;
        
/* 代码页标志 */
        
public sbyte CodePageFlag;
        
/* 保留 */
        
public byte[] Reserved2 = new byte[2];
    }

}

2。定义.dbf文件字段信息类。


namespace  Sunman.DBF
{
    
using System;

    
/// <summary>
    
/// .dbf 文件的字段信息类
    
/// </summary>

    internal class DBFField
    
{
        
public const int DBFFieldSize = 32;
        
/* 字段名称 */ 
        
public byte[] Name = new byte[11];
        
/* 字段类型 C - 字符型  
                    Y - 货币型  
                    N - 数值型  
                    F - 浮点型  
                    D - 日期型  
                    T - 日期时间型  
                    B - 双精度型  
                    I - 整型  
                    L - 逻辑型 
                    M - 备注型  
                    G - 通用型  
                    C - 字符型(二进制) 
                    M - 备注型(二进制) 
                    P - 图片型  
        
*/
 
        
public sbyte Type;
        
/* 字段偏移量 */
        
public uint Offset;
        
/* 字段长度 */ 
        
public byte Length;
        
/* 浮点数小数部分长度 */ 
        
public byte Precision;
        
/* 保留 */ 
        
public byte[] Reserved = new byte[2];
        
/* dBASE IV work area id */ 
        
public sbyte DbaseivID;
        
/* */
        
public byte[] Reserved2 = new byte[10];
        
/* */
        
public sbyte ProductionIndex;
    }

}

3。.dbf文件访问类。使用System.IO.BinaryReader类来读取.dbf文件的二进制数据,然后根据不同的字段类型和数据在.dbf文件里的存储格式对读取到的二进制数据进行解码操作。

(1)。Numeric、float、char型的数据都是明文存储,也只要江二进制数据转换为对应的字符串形式然后转换为相应的数据类型即可。

(2)。Integer型:32位。存储的就是integer类型数据的内存格式,将各个字节的数据移位然后或上即可。

(3)。Double型:64位,不定精度。应该也是double类型数据的内存格式,由于小数位数不确定,使用指针直接获取内存信息。

(4)。Currency型:64位,精度很重要(明确为4位)。读取到的数据.net中用Decimal类型存储。.dbf文件中存储的应该是Currency数据放大10000倍后的long类型数据的内存格式,所以现将文件中的二进制数据读取为对应的int64类型数据,然后缩小10000倍就行。

(5)。Date型:64位。存储的也是明文,前32位为带世纪的年度,后16位为日,剩下的16位为月。

(6)。DateTime型:64位。存储格式与Date型差异就大了。我认为前32位存储的是日期相对于某一日(还没弄清楚是那一日,只是离1年1月1日有1721426天的差距),后32位存储的是时间部分的毫秒数,我就分日期部分和时间部分读取。将日期部分读取为int32数据的天数,减去1721426后,增加到1年1月1日,得到日期部分;读取时间部分的数据为int32的毫秒数,增加到前面获取到的日期上,得到完整的日期。

获取到字段数据后,再对数据进行排错处理。这个主要是针对.dbf中numeric型和float型数据可以只存在空格和"."的现象和boolean型数据存储的是“T”和“F”。

代码如下:


namespace  Sunman.DBF
{
    
using System;
    
using System.IO;

    
/// <summary>
    
/// .dbf文件操作类
    
/// </summary>

    public class DBFFile : IDisposable
    
{
        
private const string MSG_OPEN_FILE_FAIL = "不能打开文件{0}";

        
private bool _isFileOpened;
        
private byte[] _recordBuffer;
        
private DBFField[] _dbfFields;
        
private System.IO.FileStream _fileStream = null;
        
private System.IO.BinaryReader _binaryReader = null;
        
private string _fileName = string.Empty;
        
private uint _fieldCount = 0;
        
private int _recordIndex = -1;
        
private uint _recordCount = 0;
        
private DBFHeader _dbfHeader = null;

        
/// <summary>
        
/// 构造函数
        
/// </summary>

        public DBFFile()
        
{
        }


        
/// <summary>
        
/// 构造函数
        
/// </summary>
        
/// <param name="fileName"></param>

        public DBFFile(string fileName)
        
{
            
if (null != fileName && 0 != fileName.Length)
                
this._fileName = fileName;
        }


        
/// <summary>
        
/// 清理所有正在使用的资源。
        
/// </summary>

        protected virtual void Dispose( bool disposing )
        
{
            
if( disposing )
            
{
                
this._recordBuffer = null;
                
this._dbfHeader = null;
                
this._dbfFields = null;
            
                
if (this.IsFileOpened && null != this._fileStream)
                
{
                    
this._fileStream.Close();
                    
this._binaryReader.Close();
                }

                
this._fileStream = null;
                
this._binaryReader = null;

                
this._isFileOpened = false;
                
this._fieldCount = 0;
                
this._recordCount = 0;
                
this._recordIndex = -1;
            }

        }


        
/// <summary>
        
/// 打开dbf文件
        
/// </summary>
        
/// <returns></returns>

        public bool Open()
        
{
            
try
            
{
                
return this.Open(null);
            }

            
catch (Exception e)
            
{
                
throw e;
            }

        }


        
/// <summary>
        
/// 打开dbf文件
        
/// </summary>
        
/// <param name="fileName"></param>
        
/// <returns></returns>

        public bool Open(string fileName)
        
{        
            
if (null != fileName)
                
this._fileName = fileName;

            
bool ret = false;

            
try
            
{
                
if (!this.OpenFile())
                
{
                    
// 不能打开dbf文件,抛出不能打开文件异常
                    throw new Exception(string.Format(MSG_OPEN_FILE_FAIL, this._fileName));
                }


                
// 读取文件头信息
                ret = this.ReadFileHeader();

                
// 读取所有字段信息
                if (ret)
                    ret 
= this.ReadFields();

                
// 分配记录缓冲区
                if (ret && null == this._recordBuffer)
                
{
                    
this._recordBuffer = new byte[this._dbfHeader.RecordLength];

                    
if (null == this._recordBuffer)
                        ret 
= false;
                }


                
// 如果打开文件或读取信息不成功,关闭dbf文件
                if (!ret)
                    
this.Close();
            }

            
catch (Exception e)
            
{
                
throw e;
            }


            
// 设置当前记录索引为
            this._recordIndex = -1;

            
// 返回打开文件并且读取信息的成功状态
            return ret;
        }

  • 0
    点赞
  • 9
    收藏
    觉得还不错? 一键收藏
  • 9
    评论
Introduction This article present a tiny database engine which implements fixed width record tables and BTree indexes. This library is a work in progress, I am looking for some feedback on this to see if there is a need for it. This article is also a work in progress please bookmark and come back later, if you want to see the final article. Why an embedded database Although most of us will use a SQL Server to store and retrieve data sets. There are several situation where an embedded database make sense. When you don't have a SQL Server available When you want your footprint as small as possible and can't afford SQL Express When you want to manipulate or cache SQL data When you need to write highly procedural data manipulation routines When you want maximum speed Features Despite its small size DbfDotNet provides a number of features that you might find useful Type safe In DbfDotNet you manipulate classes with native field types. All data conversion plumbing is done automatically. Very simple entity framework Creating a record and accessing its propery is only what you need. Very small memory footprint Last time I checked the dbfDotNet dll was 50Kb. Other databases are 1Mb to 10Mb. I would appreciate if someone could do some memory usage comparison (I will insert it here). Fast DbfDotNet was conceived for speed. DbfDotNet do not use PInvoke, Threading locks, and do not implement any transaction system. Those 3 technologies have a performance cost that it won't have to pay. In contrast it is using TypeSafe records (without boxing/unboxing) and type safe emitted code. The code is emitted only once per table. It has therefore I believe the potential to be the fastest embedded .Net database there is. I would appreciate if someone could do some speed comparison (I will insert it here). Very small runtime memory usage When you use in Memory DataTable or SQL requests that return DataSets, the entire result sets is in memory. DbfDotNet works conjointly with the garbage collector. As soon as you're finished modifying an entity the garbage collector will mark the record buffer to be saved to disk and released from memory. Why Dbf By default the files are compatible with dBase and can therefore be open in Excel and many other packages. I have been asked : Why Dbf ? Dbf is an old format. The answer is a bit long but simple. As I said earlier DbfDotNet is designed to be as fast as possible. In order to get the database started and get some interest I need two things: A good product A good user base I know by experience that the DBF format will appeal to some of you for several reason: You can easily backup DBF files (and leave index files) You can check DBF content using Excel and many other tools DBF is well known and simple to implement It can be extended to modern types (and has been by clipper and fox pro) Most importantly for me, implementing the .DBF rather that my own custom format has no impact on runtime speed. How does it compare to ADO.Net, SQL, SqlLite, SharpSQL ... I did some speed test against another database (which I won't name) The results are quite encouraging. Dbf.Net ADO.Net Collapse Copy CodeOpening DbfDotNetDatabase: 185 ms Insert 1000 individuals: 39 ms Read individuals sequentially: 5 ms Read individual randomly: 3 ms Modifying individuals: 21 ms Create DateOfBirth index: 77 ms Michael Simmons 22/07/1909 Mark Adams 21/09/1909 Charles Edwards 28/09/1909 ... total 1000 records Enumerate Individuals by age: 36 ms Closing DbfDotNetDatabase: 44 ms Collapse Copy CodeOpening ADO.Net Database: 459 ms Insert 1000 individuals: 80601 ms Read individuals sequentially: 1655 ms Read individual randomly: 1666 ms Modifying individuals: 75574 ms Create DateOfBirth index: 80 ms Michael Simmons 22/07/1909 Mark Adams 21/09/1909 Charles Edwards 28/09/1909 ... total 1000 records Enumerate Individuals by age: 29 ms Closing ADO.Net Database: 0 ms In this test Dbf.Net runs nearly 400 times faster. This is quite unfair however. Dbf.Net does not have transactions and is not ACID. Lets not focus to much on speed but more on code differences: Creating a Table Creating the table is quite different. Dbf.Net requires a type safe record upfront to create a table. In ADO.Net you provide a string. Dbf.Net ADO.Net Collapse Copy CodeDbfTable<dbfdotnetindividual> mIndividuals; void CreateIndividualTable() { mIndividuals = new DbfTable<dbfdotnetindividual>( @"individuals.dbf", Encoding.ASCII, DbfDotNet.DbfVersion.dBaseIV); } class Individual : DbfDotNet.DbfRecord, IIndividual { [DbfDotNet.Column(Width = 20)] public string FIRSTNAME; [DbfDotNet.Column(Width = 20)] public string MIDDLENAME; [DbfDotNet.Column(Width = 20)] public string LASTNAME; public DateTime DOB; [DbfDotNet.Column(Width = 20)] public string STATE; } Collapse Copy CodeConnection _cnn = null; void ITestDatabase.CreateIndividualTable() { _cnn = new System.Data.Connection( "Data Source=adoNetTest.db"); _cnn.Open(); using (DbCommand cmd = _cnn.CreateCommand()) { cmd.CommandText = "CREATE TABLE INDIVIDUAL (ID int primary key, FIRSTNAME VARCHAR(20), MIDDLENAME VARCHAR(20), LASTNAME VARCHAR(20), DOB DATE, STATE VARCHAR(20))"; cmd.ExecuteNonQuery(); } } Inserting new entries in a table: Dbf.Net ADO.Net Collapse Copy Codevoid InsertNewIndividual( int id, string firstname, string middlename, string lastname, DateTime dob, string state) { var indiv = mIndividuals.NewRecord(); indiv.FIRSTNAME = firstname; indiv.MIDDLENAME = middlename; indiv.LASTNAME = lastname; indiv.DOB = dob; indiv.STATE = state; indiv.SaveChanges(); } Collapse Copy Codevoid InsertNewIndividual( int id, string firstname, string middlename, string lastname, DateTime dob, string state) { using (DbCommand cmd = _cnn.CreateCommand()) { cmd.CommandText = string.Format( "INSERT INTO INDIVIDUAL (ID, FIRSTNAME, MIDDLENAME, LASTNAME, DOB, STATE) VALUES({0}, '{1}', '{2}', '{3}', '{4}', '{5}');", id, firstname, middlename, lastname, dob.ToString("yyyy-MM-dd HH:mm:ss"), state); cmd.ExecuteNonQuery(); } } Getting an individual by record ID Dbf.Net ADO.Net Collapse Copy CodeIIndividual GetIndividualById(int id) { DbfDotNetIndividual result = mIndividuals.GetRecord(id); return result; } Collapse Copy CodeIIndividual GetIndividualById(int id) { using (DbCommand cmd = _cnn.CreateCommand()) { cmd.CommandText = string.Format( "SELECT * FROM INDIVIDUAL WHERE ID=" + id); var reader = cmd.ExecuteReader(); try { if (reader.Read()) return GetNewIndividual(reader); else return null; } finally { reader.Close(); } } } Individual GetNewIndividual( DbDataReader reader) { var res = new Individual(); res.ID = reader.GetInt32(0); res.FirstName = reader.GetString(1); res.MiddleName = reader.GetString(2); res.LastName = reader.GetString(3); res.Dob = reader.GetDateTime(4); res.State = reader.GetString(5); return res; } class Individual : IIndividual { public int ID { get; set; } public string FirstName { get; set; } public string MiddleName { get; set; } public string LastName { get; set; } public DateTime Dob { get; set; } public string State { get; set; } } Saving a modified individual back to the database. In Dbf.Net you don't have to write any code, if you don't want to wait for the garbage collector to collect your individual you can call SaveChanges. Dbf.Net ADO.Net Collapse Copy Codevoid SaveIndividual( Individual individual) { individual.SaveChanges(); } Collapse Copy Codevoid SaveIndividual( IIndividual individual) { using (DbCommand cmd = _cnn.CreateCommand()) { cmd.CommandText = string.Format( "UPDATE INDIVIDUAL SET DOB='{1}' WHERE ID={0};", individual.ID, individual.Dob.ToString( "yyyy-MM-dd HH:mm:ss")); cmd.ExecuteNonQuery(); } } Creating an Index Dbf.Net ADO.Net Collapse Copy Codevoid CreateDobIndex() { var sortOrder = new DbfDotNet.SortOrder<Individual>( /*unique*/false); sortOrder.AddField("DOB"); mDobIndex = mIndividuals.GetIndex( "DOB.NDX", sortOrder); } Collapse Copy Codevoid CreateDobIndex() { using (DbCommand cmd = _cnn.CreateCommand()) { cmd.CommandText = string.Format( "CREATE INDEX DOB_IDX ON INDIVIDUAL (DOB)"); cmd.ExecuteNonQuery(); } } Getting individuals sorted by Age Dbf.Net ADO.Net Collapse Copy CodeIEnumerable<Individual> IndividualsByAge() { foreach (Individual indiv in mDobIndex) { yield return indiv; } } Collapse Copy CodeIEnumerable<Individual> IndividualsByAge() { using (DbCommand cmd = _cnn.CreateCommand()) { cmd.CommandText = string.Format( "SELECT * FROM INDIVIDUAL ORDER BY DOB"); var reader = cmd.ExecuteReader(); try { while (reader.Read()) { yield return GetNewIndividual(reader); } } finally { reader.Close(); } } } High Level Interface I have been asked how I compare to other SQL databases. Again DbfDotNet is not a SQL engine. It is rather an object persistence framework, like the Microsoft Entity Framework or NHibernate. The difference is that it doesn't translate object manipulations into SQL requests because it speaks directly to the database layer. I would love to write a proper Dbf to Linq interface, if you want to help me on this please volunteer. The difference Using the code Warning: This project is at its infancy, it has not been tested thoroughly. You can try it but please don't use it in a live environment. If you want speed however and are ready to either report or fix issues that might arrise: Create a C# project Reference DbfDotNet.dll in your project Create a record class Write some code manipulate the records Point 3 and 4 are expanded below. The DbfRecord class The DbfRecord class represent one row in your table. You can can the column attribute to change DBF specific parameters. Collapse Copy Code class Individual : DbfDotNet.DbfRecord { [Column(Width = 20)] public string FIRSTNAME; [Column(Width = 20)] public string MIDDLENAME; [Column(Width = 20)] public string LASTNAME; public DateTime DOB; [Column(Width = 20)] public string STATE; }The system automatically chooses the DbfField most appropriate for your datatype. The DbfTable class In order to store your records somewhere you need to create a Table: Collapse Copy Code individuals = new DbfTable<Individual>( @"individuals.dbf", Encoding.ASCII, DbfVersion.dBaseIV); Note that this using a type safe template. Every record in the table are individual's. Record Manipulation You can add new lines in the table by using the NewRecord Collapse Copy Code var newIndiv = individuals.NewRecord();Then you simply use the fields in your record Collapse Copy Code newIndiv.LASTNAME = "GANAYE";Optionally you can make a call to SaveChanges to immediately save your changes. If you don't the data will be saved when your individual is garbage collected. Collapse Copy Code newIndiv.SaveChanges(); Index support This is still very basic. First you define your sort order: Collapse Copy Code var sortOrder = new SortOrder<Individual>(/* unique */ false); sortOrder.AddField("LASTNAME");Then you can get your index: Collapse Copy Code mIndex = individuals.GetIndex("lastname.ndx", sortOrder); You can then, In a type safe way, retrieve any individual from your index. Collapse Copy Code individual = mIndex.GetRecord(rowNo); In order to maximize speed, the index emit its own type safe code for : reading the index fields from the DBF record reading and writing index entries comparing index entries Inner architecture DbfDotNet main class is the ClusteredFile The ClusteredFile is a wrapper around stream that provide paging and caching support. The ClusteredFile is the base class for DbfFile and NdxFile. It will also be the base class for memo files when I write them. The ClusteredFile uses a class called QuickSerializer to serialize the record content to a byte array. QuickSerializer parse the Record fields and generate a bit of IL code for every fields to allow reading, saving and comparison. NdxFile implements a B+Tree index Roadmap My plan is to keep this library extremelly small. It is not my intention to implement any transaction or multi-threading support. I will implement : support for every DBF fields types memo fields (VARCHAR type) multiple indexes files (*.mdx) Proper documentation LINQ (in a separate dll) If you want to help me on this project please contact me. Points of Interest In order to maximize speed I forced myself to not use any thread synchronization locking. Each set of Dbf + Indexes must be called from a given thread. In other word each dbf file and its index can be used by only one thread. I encountered a problem though when the Garbage Collector finalize a record, this is done in the Garbage Collector thread. I did not want to lock a resource and ended up writing this code: Collapse Copy Codeclass Record { private RecordHolder mHolder; ~Record() { try { ... } finally { mHolder.RecordFinalized.Set(); } } } Each record has a RecordHolder that store a ReadBuffer and potentially a WriteBuffer. When the record finalize it signal the RecordHolder that the record has been finalized. This instruction is not blocking, it raises a flag that can be used in other threads. Collapse Copy Codeclass ClusteredFile { internal virtual protected Record InternalGetRecord(UInt32 recordNo) { RecordHolder holder = null; if (!mRecordsByRecordNo.TryGetValue(recordNo, out holder)) {...} record = holder.mRecordWeakRef.Target; if (record==null) { // the object is not accessible it has finalized a while ago or is being finalized if (holder.RecordFinalized.WaitOne()) { //Now it has finalized we will create a new record holder.RecordFinalized.Reset(); holder.Record = OnCreateNewRecord(/*isnew*/false, recordNo); } } return holder.Record; } } And then when the table thread try to get the record while it is disposing we use the method : holder.RecordFinalized.WaitOne() to make sure the finalization has completed first. Most of the time this method won't be blocking your DBF thread as the record has been finalized some time ago. History 2009 June 4th : Added samples and ADO.Net comparison 2009 June 1st : First DbfDotNet (C#) release. 2000 May 21st : I wrote my first database engine, it is called tDbf and works on Delphi. License This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 9
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值