TOML简介及NSQD对TOML的使用

1、什么是TOML？

TOML的全称是　[Tom's Obvious Minimal Language]，其作者是 GitHub联合创始人　。

TOML的目标是成为一个极简的配置文件格式，被设计成可以无歧义地被映射为哈希表，从而被多种语言解析。

官网地址：https://toml.io/en/

2、官方示例

 

# This is a TOML document.

title = "TOML Example"

[owner]
name = "Tom Preston-Werner"
dob = 1979-05-27T07:32:00-08:00 # First class dates #日期时间是一等公民

[database]
server = "192.168.1.1"
ports = [ 8000, 8001, 8002 ]
connection_max = 5000
enabled = true

[servers]

  # 随意缩进
  [servers.alpha]
  ip = "10.0.0.1"
  dc = "eqdc10"

  [servers.beta]
  ip = "10.0.0.2"
  dc = "eqdc10"

[clients]
data = [ ["gamma", "delta"], [1, 2] ]

# 可以在数组里面换行
hosts = [
  "alpha",
  "omega"
]

1）注释

TOML认为所有的配置文件都应该支持注释

 

# This is a TOML comment

# This is a multiline
# TOML comment

2）数值

整数、浮点数、无穷大甚至NaN都支持。您可以使用科学记数法，甚至数千个分隔符

# integers
int1 = +99
int2 = 42
int3 = 0
int4 = -17

# hexadecimal with prefix `0x`
hex1 = 0xDEADBEEF
hex2 = 0xdeadbeef
hex3 = 0xdead_beef

# octal with prefix `0o`
oct1 = 0o01234567
oct2 = 0o755

# binary with prefix `0b`
bin1 = 0b11010110

# fractional 小数
float1 = +1.0
float2 = 3.1415
float3 = -0.01

# exponent 指数
float4 = 5e+22
float5 = 1e06
float6 = -2E-2

# both
float7 = 6.626e-34

# separators 分隔符
float8 = 224_617.445_991_228

# infinity
infinite1 = inf # positive infinity
infinite2 = +inf # positive infinity
infinite3 = -inf # negative infinity

# not a number
not1 = nan
not2 = +nan
not3 = -nan 

 

3） Boolean

布尔值总是小写

bool1 = true
bool2 = false

 

4）强大的字符串

有四种方式来表达字符串：基本表达方式、多行基本表达方式、字面量、多行字面量

基本表达方式：用单个双引号括起来

 

str1 = "I'm a string."
str2 = "You can \"quote\" me."
str3 = "Name\tJos\u00E9\nLoc\tSF."
str = "I'm a string. \"You can quote me\". Name\tJos\u00E9\nLocation\tSF."

任何Unicode字符都可以使用,除了这些必须转义的字符外：引号、反斜杠和tab以外的控制字符(U+0000到U+0008, U+000A到U+001F, U+007F)

为了方便起见，一些常用的字符有一个紧凑的转义序列：

\b         - backspace       (U+0008)
\t         - tab             (U+0009)
\n         - linefeed        (U+000A)
\f         - form feed       (U+000C)
\r         - carriage return (U+000D)
\"         - quote           (U+0022)
\\         - backslash       (U+005C)
\uXXXX     - unicode         (U+XXXX)
\UXXXXXXXX - unicode         (U+XXXXXXXX)

 

任何Unicode字符都可以用\uXXXX或\UXXXXXXXX格式进行转义。转义码必须是有效的Unicode标量值。

多行基本表达方式：用三个双引号括起来，并且允许换行。包括一个行尾反斜杠在自动消除任何非空格字符前面的空格。

 

str1 = """
Roses are red
Violets are blue"""

str2 = """\
  The quick brown \
  fox jumps over \
  the lazy dog.\
  """

str2 变为"The quick brown fox jumps over the lazy dog."（一个没有换行的句子）

字面量：用单个单引号括起来，不执行转义。

path = 'C:\Users\nodejs\templates'
path2 = '\\User\admin$\system32'
quoted = 'Tom "Dubs" Preston-Werner'
regex = '<\i\c*\s*>'

 

由于没有转义，所以无法在由单引号括起来的字符串中写入单引号。这就是多行字符串字面量的由来：

re = '''\d{2} apps is t[wo]o many'''
lines = '''
The first newline is
trimmed in raw strings.
All other whitespace
is preserved.
'''

 

5）日期和时间

TOML支持日期、时间和日期时间。包含和不包含时间间隔都可以。

 

# offset datetime
odt1 = 1979-05-27T07:32:00Z
odt2 = 1979-05-27T00:32:00-07:00
odt3 = 1979-05-27T00:32:00.999999-07:00

# local datetime
ldt1 = 1979-05-27T07:32:00
ldt2 = 1979-05-27T00:32:00.999999

# local date
ld1 = 1979-05-27

# local time
lt1 = 07:32:00
lt2 = 00:32:00.999999

6）键值对 Key/Value

key = "value"

 

key 和 value 之间的空格会忽略。

key,=,和value必须在同一行。

value必须是下面的一种类型：String,Integer,Float,Boolean,Offset Date-Time,Local Date-Time,Local Date,Local Time,Array,Inline Table

未指定Value是无效的：

 

key = # INVALID

键/值对之后必须有一个换行符(或EOF)：

first = "Tom" last = "Preston-Werner" # INVALID

 

7）Keys

Key中可以是基本表示、带引号和带"." 符号的

基本Key的表示只能包含ASCII字母、ASCII数字、下划线和横线(A-Za-z0-9_-)。注意，只允许由ASCII数字组成，例如1234，但是总是被解释为字符串。

 

key = "value"
bare_key = "value"
bare-key = "value"
1234 = "value"

带引号的Key：

"127.0.0.1" = "value"
"character encoding" = "value"
"ʎǝʞ" = "value"
'key2' = "value"
'quoted "value"' = "value"

 

遵循与基本字符串或字符串字面量完全相同的规则，并允许您使用更广泛的键名集合。但是，没有必要，不推荐使用。

通常情况下Key必须是非空的，但是空的带引号的键是允许的(尽管不鼓励)。

 

= "no key name"  # INVALID
"" = "blank"     # VALID but discouraged
'' = 'blank'     # VALID but discouraged

带“.”符号的Key：可以将相似的属性分组在一起

name = "Orange"
physical.color = "orange"
physical.shape = "round"
site."google.com" = true

 

转换成JSON格式，会如下展示：

{
  "name": "Orange",
  "physical": {
    "color": "orange",
    "shape": "round"
  },
  "site": {
    "google.com": true
  }
}

“.”周围的空格将会被忽略，最佳实践是不要使用任何无关的空白

fruit.name = "banana"     # this is best practice
fruit. color = "yellow"    # same as fruit.color
fruit . flavor = "banana"   # same as fruit.flavor

 

缩进被认为空格符将会被忽略。

多次定义一个键是无效的

 

# DO NOT DO THIS
name = "Tom"
name = "Pradyun"

注意带引号的Key和不带引号的Key是等价的：

# THIS WILL NOT WORK
spelling = "favorite"
"spelling" = "favourite"

 

只要没有直接定义键，您仍然可以对它和其中的名称进行写操作。

 

# This makes the key "fruit" into a table.
fruit.apple.smooth = true

# So then you can add to the table "fruit" like so:
fruit.orange = 2

# THE FOLLOWING IS INVALID

# This defines the value of fruit.apple to be an integer.
fruit.apple = 1

# But then this treats fruit.apple like it's a table.
# You can't turn an integer into a table.
fruit.apple.smooth = true

不鼓励定义无序的带点符号的Key :

# VALID BUT DISCOURAGED

apple.type = "fruit"
orange.type = "fruit"

apple.skin = "thin"
orange.skin = "thick"

apple.color = "red"
orange.color = "orange"

 

# RECOMMENDED

apple.type = "fruit"
apple.skin = "thin"
apple.color = "red"

orange.type = "fruit"
orange.skin = "thick"
orange.color = "orange"

因为最基本的Key能够用ASCII整数组成，可能带点符号的Key写成像浮点数float，不建议这样使用。

3.14159 = "pi"

 

映射成 JSON格式会是这样：

{ "3": { "14159": "pi" } }

8）Array

数组是方括号，里面有值。空格将被忽略。元素之间用逗号分隔。数组可以包含键/值对中允许的相同数据类型的值。不同类型的值可以混合使用。

integers = [ 1, 2, 3 ]
colors = [ "red", "yellow", "green" ]
nested_arrays_of_ints = [ [ 1, 2 ], [3, 4, 5] ]
nested_mixed_array = [ [ 1, 2 ], ["a", "b", "c"] ]
string_array = [ "all", 'strings', """are the same""", '''type''' ]

# Mixed-type arrays are allowed
numbers = [ 0.1, 0.2, 0.5, 1, 2, 5 ]
contributors = [
  "Foo Bar <foo@example.com>",
  { name = "Baz Qux", email = "bazqux@example.com", url = "https://example.com/bazqux" }
]

 数组可以跨多行。允许在数组的最后一个值之后使用终止逗号(也称为尾随逗号)。值、逗号和右括号之前可以有任意数量的换行符和注释。数组值和逗号之间的缩进被视为空格并被忽略。

integers2 = [
  1, 2, 3
]

integers3 = [
  1,
  2, # this is ok
]

9）Table

表(也称为哈希表或字典)是键/值对的集合。它们由标题定义，在一行中单独使用方括号。你可以区分头文件和数组，因为数组永远只是值。

[table]

 

在它下面，直到下一个头或EOF，是该表的键/值。表中的键/值对不能保证有任何特定的顺序。

 

[table-1]
key1 = "some string"
key2 = 123

[table-2]
key1 = "another string"
key2 = 456

Table的命名规则与Key的命名规则相同:

[dog."tater.man"]
type.name = "pug"

 

转换成的JSON格式如下：

 

{ "dog": { "tater.man": { "type": { "name": "pug" } } } }

Key周围的空格被忽略。然而，最佳实践是不要使用任何无关的空白。

[a.b.c]            # this is best practice
[ d.e.f ]          # same as [d.e.f]
[ g .  h  . i ]    # same as [g.h.i]
[ j . "ʞ" . 'l' ]  # same as [j."ʞ".'l']

 

缩进被视为空白并被忽略。

你不需要指定所有的超级表。TOML可以自动处理。

 

# [x] you
# [x.y] don't
# [x.y.z] need these
[x.y.z.w] # for this to work

[x] # defining a super-table afterward is ok

 

空表是允许的，并且其中没有键/值对。

和Key一样，一个表不能定义多次。这样做是无效的。

 

# DO NOT DO THIS

[fruit]
apple = "red"

[fruit]
orange = "orange"

# DO NOT DO THIS EITHER

[fruit]
apple = "red"

[fruit.apple]
texture = "smooth"

 

不鼓励乱序定义表。

 

# VALID BUT DISCOURAGED
[fruit.apple]
[animal]
[fruit.orange]

# RECOMMENDED
[fruit.apple]
[fruit.orange]
[animal]

顶层表，也称为根表（root table），从文档的开头开始，在第一个表头(或EOF)之前结束。与其他表不同，它是无名称的，不能被重新定位。

# Top-level table begins.
name = "Fido"
breed = "pug"

# Top-level table ends.
[owner]
name = "Regina Dogman"
member_since = 1999-08-04

带点“.” 的Key为最后一个键部分之前的每个键部分创建并定义一个表，前提是以前没有创建这样的表

fruit.apple.color = "red"
# Defines a table named fruit
# Defines a table named fruit.apple

fruit.apple.taste.sweet = true
# Defines a table named fruit.apple.taste
# fruit and fruit.apple were already created

 

由于表不能被多次定义，所以不允许使用[table]头文件重新定义这些表。同样，也不允许使用带点“.”的key重新定义已经在[table]形式中定义的表。然而，[table]表单可以用来在通过带“.”的Key定义的表中定义子表。

 

 

[fruit]
apple.color = "red"
apple.taste.sweet = true

# [fruit.apple]  # INVALID
# [fruit.apple.taste]  # INVALID

[fruit.apple.texture]  # you can add sub-tables
smooth = true

 

10）Inline Table

内联表为表示表提供了更紧凑的语法。它们对于分组数据特别有用，否则这些数据很快就会变得冗长。内联表完全定义在花括号:{和}中。在大括号内，可以出现零个或多个以逗号分隔的键/值对。键/值对的形式与标准表中的键/值对相同。允许所有值类型，包括内联表。

内联表的目的是显示在一行上。行内表的最后一个键/值对之后不允许使用终止逗号(也称为尾随逗号)。大括号之间不允许换行，除非它们在值内有效。即使如此，也不鼓励将内联表分成多个行。如果你发现自己被这种欲望所控制，这意味着你应该使用标准的表格。

 

name = { first = "Tom", last = "Preston-Werner" }
point = { x = 1, y = 2 }
animal = { type.name = "pug" }

上面的内联表与下面的标准表定义相同:

[name]
first = "Tom"
last = "Preston-Werner"

[point]
x = 1
y = 2

[animal]
type.name = "pug"

 

内联表是完全自包含的，并定义其中的所有键和子表。键和子表不能添加到大括号之外。

 

[product]
type = { name = "Nail" }
# type.edible = false  # INVALID

类似地，内联表也不能用于向已定义的表添加键或子表。

[product]
type.name = "Nail"
# type = { edible = false }  # INVALID

 

11) Array of Tables

该头文件的第一个实例定义数组及其第一个表元素，每个后续实例在该数组中创建并定义一个新的表元素。表按照遇到的顺序插入到数组中:

 

[[products]]
name = "Hammer"
sku = 738594937

[[products]]  # empty table within the array

[[products]]
name = "Nail"
sku = 284758393

color = "gray"

转换成的JSON格式如下：

{
  "products": [
    { "name": "Hammer", "sku": 738594937 },
    { },
    { "name": "Nail", "sku": 284758393, "color": "gray" }
  ]
}

 

对表数组的任何引用都指向数组中最近定义的表元素。这允许您在最近的表中定义子表，甚至表的子数组。

 

[[fruits]]
name = "apple"

[fruits.physical]  # subtable
color = "red"
shape = "round"

[[fruits.varieties]]  # nested array of tables
name = "red delicious"

[[fruits.varieties]]
name = "granny smith"


[[fruits]]
name = "banana"

[[fruits.varieties]]
name = "plantain"

 

转换成的格式如下：

 

{
  "fruits": [
    {
      "name": "apple",
      "physical": {
        "color": "red",
        "shape": "round"
      },
      "varieties": [
        { "name": "red delicious" },
        { "name": "granny smith" }
      ]
    },
    {
      "name": "banana",
      "varieties": [
        { "name": "plantain" }
      ]
    }
  ]
}

如果表或表的数组的父元素是数组元素，则必须在定义子元素之前已经定义了该元素。试图反转该顺序必须在解析时产生错误。

# INVALID TOML DOC
[fruit.physical]  # subtable, but to which parent element should it belong?
color = "red"
shape = "round"

[[fruit]]  # parser must throw an error upon discovering that "fruit" is
           # an array rather than a table
name = "apple"

 

尝试追加静态定义的数组，即使该数组为空，也必须在解析时产生错误。

# INVALID TOML DOC
fruits = []

[[fruits]] # Not allowed

 

试图定义与已经建立的数组同名的普通表，在解析时必须产生错误。尝试将普通表重新定义为数组也同样会产生解析时间错误。

# INVALID TOML DOC
[[fruits]]
name = "apple"

[[fruits.varieties]]
name = "red delicious"

# INVALID: This table conflicts with the previous array of tables
[fruits.varieties]
name = "granny smith"

[fruits.physical]
color = "red"
shape = "round"

# INVALID: This array of tables conflicts with the previous table
[[fruits.physical]]
color = "green"

您也可以在适当的地方使用内联表：

points = [ { x = 1, y = 2, z = 3 },
           { x = 7, y = 8, z = 9 },
           { x = 2, y = 4, z = 8 } ]

 

12）文件扩展名

TOML文件应该使用.toml扩展名

13）MIME Type

当在因特网上传输TOML文件时，适当的MIME类型是application/ toml

2、NSQD对TOML的使用

 

1) 第三方库：

 

或者是单独Get下来：

go get github.com/BurntSushi/toml

 

2) 创建example.toml

 

# This is a TOML document. Boom.

title = "TOML Example"

[owner]
name = "Tom Preston-Werner"
organization = "GitHub"
bio = "GitHub Cofounder & CEO\nLikes tater tots and beer."
dob = 1979-05-27T07:32:00Z # First class dates? Why not?

[database]
server = "192.168.1.1"
ports = [ 8001, 8001, 8002 ]
connection_max = 5000
enabled = true

[servers]

# You can indent as you please. Tabs or spaces. TOML don't care.
[servers.alpha]
ip = "10.0.0.1"
dc = "eqdc10"

[servers.beta]
ip = "10.0.0.2"
dc = "eqdc10"

[clients]
data = [ ["gamma", "delta"], [1, 2] ] # just an update to make sure parsers support it

# Line breaks are OK when inside arrays
hosts = [
    "alpha",
    "omega"
]

2、分析这个toml文件，创建Table与Struct的对应关系：

TOML的Table	Go Struct
ConfigFile	tomlConfig
owner	ownerInfo
database	database
servers	server
servers.alpha	server
servers.beta	server
clients	clients

分析代码整个文件的tomlConfig struct应有的属性:

• Title
• ownerInfo
• database
• map[string]server,因为Table本身就是键值对的集合。
• clients

clients struct应有的属性为：

• Data [][]interface{}：二维数组
• Hosts []string： 数组

server因为根据Key值进行区分，因此server struct应有的属性为：

• IP string
• DC string

 

 

整体的结构体的代码如下：

type tomlConfig struct {
   Title string
   Owner ownerInfo
   DB database `toml:"database"`
   Servers map[string]server
   Clients clients
}
type server struct {
   IP string
   DC string
}

type database struct {
   Server string
   Ports []int
   ConnMax int `toml:"connection_max"`
   Enabled bool
}

type clients struct {
   Data [][]interface{}
   Hosts []string
}
type ownerInfo struct {
   Name string
   Org string `toml:"organization"`
   Bio string
   DOB time.Time
}

 

解析

func main() {
   //配置文件结构体变量
   var config tomlConfig
   // mapping  TOML values to Go values
   if _, err := toml.DecodeFile("interview/toml/example.toml", &config); err != nil {
      fmt.Println(err)
      return
   }

   fmt.Printf("Title： %s\n",config.Title)
   fmt.Printf("Owner: %s (%s,%s),Born: %s\n",config.Owner.Name,config.Owner.Org,config.Owner.Bio,config.Owner.DOB)
   fmt.Printf("Database: %s %v (Max conn. %d),Enabled? %v\n",config.DB.Server,config.DB.Ports,config.DB.ConnMax,config.DB.Enabled)
   for serverName,server := range config.Servers {
      fmt.Printf("Server: %s,(%s,%s)\n",serverName,server.IP,server.DC)
   }
   fmt.Printf("Client data: %v\n",config.Clients.Data)
   fmt.Printf("Client hosts: %v\n",config.Clients.Hosts)
}

 

输出结果：

Title： TOML Example
Owner: Tom Preston-Werner (GitHub,GitHub Cofounder & CEO
Likes tater tots and beer.),Born: 1979-05-27 07:32:00 +0000 UTC
Database: 192.168.1.1 [8001 8001 8002] (Max conn. 5000),Enabled? true
Server: alpha,(10.0.0.1,eqdc10)
Server: beta,(10.0.0.2,eqdc10)
Client data: [[gamma delta] [1 2]]
Client hosts: [alpha omega]

主要是利用了以下方法：

// DecodeFile is just like Decode, except it will automatically read the
// contents of the file at `fpath` and decode it for you.
func DecodeFile(fpath string, v interface{}) (MetaData, error)

最终是利用反射进行文件的解析与映射，主要核心代码如下：

 

func Decode(data string, v interface{}) (MetaData, error) {
   rv := reflect.ValueOf(v)
   if rv.Kind() != reflect.Ptr {
      return MetaData{}, e("Decode of non-pointer %s", reflect.TypeOf(v))
   }
   if rv.IsNil() {
      return MetaData{}, e("Decode of nil %s", reflect.TypeOf(v))
   }
   p, err := parse(data)
   if err != nil {
      return MetaData{}, err
   }
   md := MetaData{
      p.mapping, p.types, p.ordered,
      make(map[string]bool, len(p.ordered)), nil,
   }
   return md, md.unify(p.mapping, indirect(rv))
}