经验分享-大小写敏感相关问题

一 操作系统

背景

在一次开发中(Go),项目本地编译通过并且运行无异常。当将代码上传的线上机器进行编译,加载导入的包时报错

no matching versions for query "latest"

猜测

由于本地编译无问题,所以猜测

  1. 权限问题(无权限拉取此仓库)
  2. 代理问题(代理不对,找不到此仓库)
  3. 项目路径问题(项目路径不对,找不到此仓库)

验证

去掉无法加载的包,其他类似的包都能正常加载,所以上面三个猜测均不成立

定位

换一种思路,看看这个包在别人的项目是怎么使用的。通过一个字母一个字母的对比,发现我import的包名含有大写字母。去掉大写字母后本地和线上编译运行全部正常

原因

Windows、macOS 默认大小写不敏感

Linux 默认大小写敏感

此例中 本地操作系统是macOS,线上操作系统是Linux

备注

二 HTTP协议

背景

最近一次开发中,从 HTTP 请求 Header 中取参数 x-y 时,发现发起请求时传了 x-y ,但是服务端却取不到,导致系统报错。打印日志,发现服务端收到的请求Header中,有一个 X-Y 字段。

定位

x-y 参数在网络传输的某个环节被转换成了 X-Y 

验证

1. 在哪里被转换了?

一个使用Go编写的流量转发模块做的转换。

req, _ := http.NewRequest()
req.Header.Add()

// Add adds the key, value pair to the header.
// It appends to any existing values associated with key.
// The key is case insensitive; it is canonicalized by
// CanonicalHeaderKey.
func (h Header) Add(key, value string) {
	textproto.MIMEHeader(h).Add(key, value)
}

// CanonicalMIMEHeaderKey returns the canonical format of the
// MIME header key s. The canonicalization converts the first
// letter and any letter following a hyphen to upper case;
// the rest are converted to lowercase. For example, the
// canonical key for "accept-encoding" is "Accept-Encoding".
// MIME header keys are assumed to be ASCII only.
// If s contains a space or invalid header field bytes, it is
// returned without modifications.
func CanonicalMIMEHeaderKey(s string) string {
	commonHeaderOnce.Do(initCommonHeader)

	// Quick check for canonical encoding.
	upper := true
	for i := 0; i < len(s); i++ {
		c := s[i]
		if !validHeaderFieldByte(c) {
			return s
		}
		if upper && 'a' <= c && c <= 'z' {
			return canonicalMIMEHeaderKey([]byte(s))
		}
		if !upper && 'A' <= c && c <= 'Z' {
			return canonicalMIMEHeaderKey([]byte(s))
		}
		upper = c == '-'
	}
	return s
}

翻译后:

        CanonicalMIMEHeaderKey 返回 MIME 标头键 s 的规范格式。规范化将第一个字母和连字符后的任何字母转换为大写;其余的都转换为小写。例如,“accept-encoding”的规范键是“Accept-Encoding”。 MIME 标头键被假定为仅 ASCII。如果 s 包含空格或无效的头字段字节,则不加修改地返回。

流量转发模块如果使用了这个方法,那么字段名就会被转换。

2. 这个转换是否合理?

合理! 可以看下stack overflow这个提问。

Are HTTP headers case-sensitive? - Stack Overflowhttps://stackoverflow.com/questions/5258977/are-http-headers-case-sensitive/5259004部分回答(翻译后):

 

扩展

1. HTTP协议其他元素大小写敏感吗?

再补充另两个提问:Should URL be case sensitive? - Stack Overflowhttps://stackoverflow.com/questions/7996919/should-url-be-case-sensitive/17113291#17113291

asp.net web api - Should a REST API be case sensitive or non case sensitive? - Stack Overflowhttps://stackoverflow.com/questions/21001455/should-a-rest-api-be-case-sensitive-or-non-case-sensitive/21001644#21001644 

        总计下来:

                method区分

                协议和域名不区分

                path原则上区分,但是如果站点部署在windows操作系统上,可能不会区分。

                query区分

2. 当然,我们不能轻信别人的回答,我们怎么做出验证呢?

  • 阅读RFC文档
  • 使用编程语言验证

        以HTTP/1.1为例,上面的几个回答都提到了 RFC 7230

        怎么确定这个标准是我们要找的?

                很难,RFC阅读方式:How to Read an RFC,试了下,根本搜不到。

        我们姑且认为他们说的是对的,那我们的代码是不是这样做的呢?

                以gin框架为例,其中readRequest是解析http请求的底层函数,其中除了schema,其他元素并没有强制大小写转换的逻辑,无论是接收http请求,还是发起http请求,都没有强制转换。

func readRequest(b *bufio.Reader, deleteHostHeader bool) (req *Request, err error) {
	tp := newTextprotoReader(b)
	req = new(Request)

	// First line: GET /index.html HTTP/1.0
	var s string
	if s, err = tp.ReadLine(); err != nil {
		return nil, err
	}
	defer func() {
		putTextprotoReader(tp)
		if err == io.EOF {
			err = io.ErrUnexpectedEOF
		}
	}()

	var ok bool
	req.Method, req.RequestURI, req.Proto, ok = parseRequestLine(s)
	if !ok {
		return nil, &badStringError{"malformed HTTP request", s}
	}
	if !validMethod(req.Method) {
		return nil, &badStringError{"invalid method", req.Method}
	}
	rawurl := req.RequestURI
	if req.ProtoMajor, req.ProtoMinor, ok = ParseHTTPVersion(req.Proto); !ok {
		return nil, &badStringError{"malformed HTTP version", req.Proto}
	}

	// CONNECT requests are used two different ways, and neither uses a full URL:
	// The standard use is to tunnel HTTPS through an HTTP proxy.
	// It looks like "CONNECT www.google.com:443 HTTP/1.1", and the parameter is
	// just the authority section of a URL. This information should go in req.URL.Host.
	//
	// The net/rpc package also uses CONNECT, but there the parameter is a path
	// that starts with a slash. It can be parsed with the regular URL parser,
	// and the path will end up in req.URL.Path, where it needs to be in order for
	// RPC to work.
	justAuthority := req.Method == "CONNECT" && !strings.HasPrefix(rawurl, "/")
	if justAuthority {
		rawurl = "http://" + rawurl
	}

	if req.URL, err = url.ParseRequestURI(rawurl); err != nil {
		return nil, err
	}

	if justAuthority {
		// Strip the bogus "http://" back off.
		req.URL.Scheme = ""
	}

	// Subsequent lines: Key: value.
	mimeHeader, err := tp.ReadMIMEHeader()
	if err != nil {
		return nil, err
	}
	req.Header = Header(mimeHeader)

	// RFC 7230, section 5.3: Must treat
	//	GET /index.html HTTP/1.1
	//	Host: www.google.com
	// and
	//	GET http://www.google.com/index.html HTTP/1.1
	//	Host: doesntmatter
	// the same. In the second case, any Host line is ignored.
	req.Host = req.URL.Host
	if req.Host == "" {
		req.Host = req.Header.get("Host")
	}
	if deleteHostHeader {
		delete(req.Header, "Host")
	}

	fixPragmaCacheControl(req.Header)

	req.Close = shouldClose(req.ProtoMajor, req.ProtoMinor, req.Header, false)

	err = readTransfer(req, b)
	if err != nil {
		return nil, err
	}

	if req.isH2Upgrade() {
		// Because it's neither chunked, nor declared:
		req.ContentLength = -1

		// We want to give handlers a chance to hijack the
		// connection, but we need to prevent the Server from
		// dealing with the connection further if it's not
		// hijacked. Set Close to ensure that:
		req.Close = true
	}
	return req, nil
}

       schema的小写转换

// parse parses a URL from a string in one of two contexts. If
// viaRequest is true, the URL is assumed to have arrived via an HTTP request,
// in which case only absolute URLs or path-absolute relative URLs are allowed.
// If viaRequest is false, all forms of relative URLs are allowed.
func parse(rawurl string, viaRequest bool) (*URL, error) {
	var rest string
	var err error

	if stringContainsCTLByte(rawurl) {
		return nil, errors.New("net/url: invalid control character in URL")
	}

	if rawurl == "" && viaRequest {
		return nil, errors.New("empty url")
	}
	url := new(URL)

	if rawurl == "*" {
		url.Path = "*"
		return url, nil
	}

	// Split off possible leading "http:", "mailto:", etc.
	// Cannot contain escaped characters.
	if url.Scheme, rest, err = getscheme(rawurl); err != nil {
		return nil, err
	}
	url.Scheme = strings.ToLower(url.Scheme)

	if strings.HasSuffix(rest, "?") && strings.Count(rest, "?") == 1 {
		url.ForceQuery = true
		rest = rest[:len(rest)-1]
	} else {
		rest, url.RawQuery = split(rest, '?', true)
	}

	if !strings.HasPrefix(rest, "/") {
		if url.Scheme != "" {
			// We consider rootless paths per RFC 3986 as opaque.
			url.Opaque = rest
			return url, nil
		}
		if viaRequest {
			return nil, errors.New("invalid URI for request")
		}

		// Avoid confusion with malformed schemes, like cache_object:foo/bar.
		// See golang.org/issue/16822.
		//
		// RFC 3986, §3.3:
		// In addition, a URI reference (Section 4.1) may be a relative-path reference,
		// in which case the first path segment cannot contain a colon (":") character.
		colon := strings.Index(rest, ":")
		slash := strings.Index(rest, "/")
		if colon >= 0 && (slash < 0 || colon < slash) {
			// First path segment has colon. Not allowed in relative URL.
			return nil, errors.New("first path segment in URL cannot contain colon")
		}
	}

	if (url.Scheme != "" || !viaRequest && !strings.HasPrefix(rest, "///")) && strings.HasPrefix(rest, "//") {
		var authority string
		authority, rest = split(rest[2:], '/', false)
		url.User, url.Host, err = parseAuthority(authority)
		if err != nil {
			return nil, err
		}
	}
	// Set Path and, optionally, RawPath.
	// RawPath is a hint of the encoding of Path. We don't want to set it if
	// the default escaping of Path is equivalent, to help make sure that people
	// don't rely on it in general.
	if err := url.setPath(rest); err != nil {
		return nil, err
	}
	return url, nil
}

        通过Postman发送请求尝试:

                query:大小写敏感,不完全一致就取不到参数

                path:大小写敏感,但是在本地macOS操作系统上,可以正常访问到目标资源

                method:大小写敏感,不一致就会报400

                域名:不敏感       

                Schema:不敏感

  • 0
    点赞
  • 0
    收藏
  • 打赏
    打赏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
©️2022 CSDN 皮肤主题:1024 设计师:我叫白小胖 返回首页
评论

打赏作者

无名小彤

你的鼓励将是我创作的最大动力

¥2 ¥4 ¥6 ¥10 ¥20
输入1-500的整数
余额支付 (余额:-- )
扫码支付
扫码支付:¥2
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、C币套餐、付费专栏及课程。

余额充值