UnixFS
UnixFs is a unix-like filesystem utilities on top of an ipld merkledag
-
io
The io subpackage provides helpers for reading files and manipulating directories. The DagReader takes a reference to a unixfs file and returns a file handle that can be read from and seeked through. The Directory interface allows you to easily read items in a directory, add items to a directory, and do lookups. -
mod
The mod subpackage implements a DagModifier type that can be used to write to an existing unixfs file, or create a new one. The logic for this is significantly more complicated than for the dagreader, so its a separate type. (TODO: maybe it still belongs in the io subpackage though?) -
hamt
The hamt subpackage implements a CHAMP hamt that is used in unixfs directory sharding. -
archive
The archive subpackage implements a tar importer and exporter. The objects created here are not officially unixfs, but in the future, this may be integrated more directly.
UnixFs file from a IpldNode
func NewUnixfsFile(ctx context.Context, dserv ipld.DAGService, nd ipld.Node) (files.Node, error) {
switch dn := nd.(type) {
case *dag.ProtoNode:
fsn, err := ft.FSNodeFromBytes(dn.Data())
if err != nil {
return nil, err
}
if fsn.IsDir() {
return newUnixfsDir(ctx, dserv, dn)
}
if fsn.Type() == ft.TSymlink {
return files.NewLinkFile(string(fsn.Data()), nil), nil
}
case *dag.RawNode:
default:
return nil, errors.New("unknown node type")
}
dr, err := uio.NewDagReader(ctx, nd, dserv)
if err != nil {
return nil, err
}
return &ufsFile{
DagReader: dr,
}, nil
}
Conversion from a block to a file
- Block format is checked in ipld.Decode. 3 formats are registered so far:
func init() {
ipld.Register(cid.DagProtobuf, DecodeProtobufBlock)
ipld.Register(cid.Raw, DecodeRawBlock)
ipld.Register(cid.DagCBOR, ipldcbor.DecodeBlock)
}
MFS
mutable IPFS filesystem. Root is initialized as:
dsk := datastore.NewKey("/local/filesroot")
rootDS.Put(dsk, c.Bytes());
// Files loads persisted MFS root
func Files(mctx helpers.MetricsCtx, lc fx.Lifecycle, repo repo.Repo, dag format.DAGService) (*mfs.Root, error) {
dsk := datastore.NewKey("/local/filesroot")
pf := func(ctx context.Context, c cid.Cid) error {
rootDS := repo.Datastore()
if err := rootDS.Sync(blockstore.BlockPrefix); err != nil {
return err
}
if err := rootDS.Sync(filestore.FilestorePrefix); err != nil {
return err
}
if err := rootDS.Put(dsk, c.Bytes()); err != nil {
return err
}
return rootDS.Sync(dsk)
}
var nd *merkledag.ProtoNode
val, err := repo.Datastore().Get(dsk)
ctx := helpers.LifecycleCtx(mctx, lc)
switch {
case err == datastore.ErrNotFound || val == nil:
nd = unixfs.EmptyDirNode()
err := dag.Add(ctx, nd)
if err != nil {
return nil, fmt.Errorf("failure writing to dagstore: %s", err)
}
case err == nil:
c, err := cid.Cast(val)
if err != nil {
return nil, err
}
rnd, err := dag.Get(ctx, c)
if err != nil {
return nil, fmt.Errorf("error loading filesroot from DAG: %s", err)
}
pbnd, ok := rnd.(*merkledag.ProtoNode)
if !ok {
return nil, merkledag.ErrNotProtobuf
}
nd = pbnd
default:
return nil, err
}
root, err := mfs.NewRoot(ctx, dag, nd, pf)
lc.Append(fx.Hook{
OnStop: func(ctx context.Context) error {
return root.Close()
},
})
return root, err
}
Conversion from a MFS path to a file
mfs.Lookup will iterate the mfs.Root to get the path. mfs.Root is created when mfs initialized.
mfsRoot
QmbxJou6T8R7b6xuqEC5jR8hDJa9EqEEVkYFfskawQDvar is the mfs.Root block, created in the begining. We can see this block actually contain some links to the files/directories…
ipfs object get QmbxJou6T8R7b6xuqEC5jR8hDJa9EqEEVkYFfskawQDvar
{“Links”:[{“Name”:“e4”,“Hash”:“QmVD5F2sxesn57H7uCJtmKMdzdMuzJ67FgXiVhaeoBt5TA”,“Size”:23598},{“Name”:“f2”,“Hash”:“QmRFNu2WqHiKp2nLZ8qMcs1ZMMps8EGEeFP3acvuYgDFay”,“Size”:74},{“Name”:“f3”,“Hash”:“QmW9BbU9Tiq8dMeoANVeSrSAevawHGDNQVbidpKFToSwZg”,“Size”:74},{“Name”:“f4”
,“Hash”:“QmRFNu2WqHiKp2nLZ8qMcs1ZMMps8EGEeFP3acvuYgDFay”,“Size”:74},{“Name”:“f5”,“Hash”:“QmRFNu2WqHiKp2nLZ8qMcs1ZMMps8EGEeFP3acvuYgDFay”,“Size”:74},{“Name”:“f6”,“Hash”:“QmW9BbU9Tiq8dMeoANVeSrSAevawHGDNQVbidpKFToSwZg”,“Size”:74},{“Name”:“file”,“Hash”:“QmRFNu2WqHiKp2
nLZ8qMcs1ZMMps8EGEeFP3acvuYgDFay”,“Size”:74},{“Name”:“test1”,“Hash”:“QmTSMdpEMakBGdamtWqzApdj4zQdAeA9NdiFn1DSv51Knu”,“Size”:70947}],“Data”:"\u0008\u0001"}
ipfs files ls
e4
f2
f3
f4
f5
f6
file
test1
Samples
Take a look at at a MFS file:
ipfs files read /f2
“hello world”
Check out the CID of this MFS file:
ipfs files stat /f2
QmRFNu2WqHiKp2nLZ8qMcs1ZMMps8EGEeFP3acvuYgDFay
Size: 16
CumulativeSize: 74
ChildBlocks: 1
Type: file
Check out the block of the CID, it has a link pointing to a block:
ipfs dag get QmRFNu2WqHiKp2nLZ8qMcs1ZMMps8EGEeFP3acvuYgDFay
{“data”:“CAIYECAQ”,“links”:[{“Cid”:{"/":“QmQX9Q2YhruUmdFR2wmUftRC9GSb8mwsh3idUSUW2AkpQ1”},“Name”:"",“Size”:24}]}
This block has the data, json encoded:
ipfs dag get QmQX9Q2YhruUmdFR2wmUftRC9GSb8mwsh3idUSUW2AkpQ1
{“data”:“CAASECJoZWxsbyB3b3JsZCIgDQoYEA==”,“links”:[]}
or, text format:
ipfs object get QmQX9Q2YhruUmdFR2wmUftRC9GSb8mwsh3idUSUW2AkpQ1
{“Links”:[],“Data”:"\u0008\u0000\u0012\u0010"hello world" \r\n\u0018\u0010"}