goproxy 源碼分析
go get 取包原理
1 第一步，正則匹配出依賴包的查詢路徑
go get可以指定具體包的import路徑或者通過其自行分析代碼中的import得出需要獲取包的路徑。但是import路徑，并不直接就是該包的查詢路徑。在go get的源碼實(shí)現(xiàn)中，包的查詢路徑是通過一組正則匹配出來的。也就是說，import路徑是必須匹配這組正則表達(dá)式的，如果不匹配的話，代碼是肯定無法編譯的。
再結(jié)合go-get參數(shù)，向遠(yuǎn)端VCS系統(tǒng)發(fā)起https://github.com/goproxyio/goproxy?go-get=1請求。
2 第二步，查詢得出包的遠(yuǎn)端倉庫地址
包的遠(yuǎn)端倉庫地址，可以通過go get請求的響應(yīng)中的go-import的meta標(biāo)簽中的content中獲取的。
3 第三步，根據(jù)倉庫地址clone到本地
雖然版本控制系統(tǒng)VCS本身就存在各類區(qū)別，但是一些基礎(chǔ)操作大多類似。在go get中具體clone的過程會(huì)根據(jù)具體的VCS采用對應(yīng)的操作。
go get 代理取包流程
了解了go get取包的基礎(chǔ)流程后，說說Go Module功能開啟后的完整流程。
可以用go get -x 查看拉取的詳細(xì)過程
go get -x github.com/goproxyio/goproxy # get https://goproxy.cn/github.com/goproxyio/@v/list# get https://goproxy.cn/github.com/@v/list# get https://goproxy.cn/github.com/goproxyio/goproxy/@v/list# get https://goproxy.cn/github.com/@v/list: 404 Not Found (0.686s)# get https://goproxy.cn/github.com/goproxyio/@v/list: 404 Not Found (0.754s)# get https://goproxy.cn/github.com/goproxyio/goproxy/@v/list: 200 OK (0.855s)# get https://goproxy.cn/github.com/goproxyio/goproxy/@v/v1.0.0.info# get https://goproxy.cn/github.com/goproxyio/goproxy/@v/v1.0.0.info: 200 OK (0.117s)go: downloading github.com/goproxyio/goproxy v1.0.0# get https://goproxy.cn/github.com/goproxyio/goproxy/@v/v1.0.0.zip# get https://goproxy.cn/github.com/goproxyio/goproxy/@v/v1.0.0.zip: 200 OK (0.228s)# get https://goproxy.cn/sumdb/sum.golang.org/supported# get https://goproxy.cn/sumdb/sum.golang.org/supported: 200 OK (0.032s)# get https://goproxy.cn/sumdb/sum.golang.org/lookup/github.com/goproxyio/[email protected]# get https://goproxy.cn/sumdb/sum.golang.org/lookup/github.com/goproxyio/[email protected]: 200 OK (0.414s)# get https://goproxy.cn/sumdb/sum.golang.org/tile/8/0/x014/109# get https://goproxy.cn/sumdb/sum.golang.org/tile/8/0/x014/199.p/195# get https://goproxy.cn/sumdb/sum.golang.org/tile/8/1/055.p/119# get https://goproxy.cn/sumdb/sum.golang.org/tile/8/0/x014/109: 200 OK (0.028s)# get https://goproxy.cn/sumdb/sum.golang.org/tile/8/1/055.p/119: 200 OK (0.040s)# get https://goproxy.cn/sumdb/sum.golang.org/tile/8/0/x014/199.p/195: 200 OK (0.057s)# get https://goproxy.cn/sumdb/sum.golang.org/tile/8/0/324# get https://goproxy.cn/sumdb/sum.golang.org/tile/8/0/324: 200 OK (0.226s)go: github.com/goproxyio/goproxy upgrade => v1.0.0# get https://goproxy.cn/github.com/goproxyio/goproxy/@v/v1.0.0.mod# get https://goproxy.cn/github.com/goproxyio/goproxy/@v/v1.0.0.mod: 200 OK (0.093s)go: finding module for package github.com/goproxyio/goproxy/internal/module# get https://goproxy.cn/github.com/goproxyio/goproxy/internal/module/@v/list# get https://goproxy.cn/github.com/goproxyio/goproxy/internal/@v/listgo: finding module for package github.com/goproxyio/goproxy/internal/cfg# get https://goproxy.cn/github.com/goproxyio/goproxy/internal/cfg/@v/listgo: finding module for package github.com/goproxyio/goproxy/internal/modfetch# get https://goproxy.cn/github.com/goproxyio/goproxy/internal/modfetch/@v/listgo: finding module for package github.com/goproxyio/goproxy/internal/modload# get https://goproxy.cn/github.com/goproxyio/goproxy/internal/modload/@v/listgo: finding module for package github.com/goproxyio/goproxy/internal/modfetch/codehost# get https://goproxy.cn/github.com/goproxyio/goproxy/internal/modfetch/codehost/@v/list# get https://goproxy.cn/github.com/goproxyio/goproxy/internal/module/@v/list: 404 Not Found (2.579s)# get https://goproxy.cn/github.com/goproxyio/goproxy/internal/modfetch/codehost/@v/list: 404 Not Found (2.474s)# get https://goproxy.cn/github.com/goproxyio/goproxy/internal/modfetch/@v/list: 404 Not Found (2.882s)# get https://goproxy.cn/github.com/goproxyio/goproxy/internal/@v/list: 404 Not Found (2.984s)# get https://goproxy.cn/github.com/goproxyio/goproxy/internal/cfg/@v/list: 404 Not Found (3.339s)# get https://goproxy.cn/github.com/goproxyio/goproxy/internal/modload/@v/list: 404 Not Found (3.353s)go: finding module for package github.com/goproxyio/goproxy/internal/modloadgo: finding module for package github.com/goproxyio/goproxy/internal/modulego: finding module for package github.com/goproxyio/goproxy/internal/modfetch/codehostgo: finding module for package github.com/goproxyio/goproxy/internal/cfggo: finding module for package github.com/goproxyio/goproxy/internal/modfetch../../../../pkg/mod/github.com/goproxyio/goproxy@v1.0.0/pkg/proxy/proxy.go:12:2: module github.com/goproxyio/goproxy@latest found (v1.0.0), but does not contain package github.com/goproxyio/goproxy/internal/cfg../../../../pkg/mod/github.com/goproxyio/goproxy@v1.0.0/pkg/proxy/proxy.go:13:2: module github.com/goproxyio/goproxy@latest found (v1.0.0), but does not contain package github.com/goproxyio/goproxy/internal/modfetch../../../../pkg/mod/github.com/goproxyio/goproxy@v1.0.0/pkg/proxy/proxy.go:14:2: module github.com/goproxyio/goproxy@latest found (v1.0.0), but does not contain package github.com/goproxyio/goproxy/internal/modfetch/codehost../../../../pkg/mod/github.com/goproxyio/goproxy@v1.0.0/pkg/proxy/proxy.go:15:2: module github.com/goproxyio/goproxy@latest found (v1.0.0), but does not contain package github.com/goproxyio/goproxy/internal/modload../../../../pkg/mod/github.com/goproxyio/goproxy@v1.0.0/pkg/proxy/proxy.go:16:2: module github.com/goproxyio/goproxy@latest found (v1.0.0), but does not contain package github.com/goproxyio/goproxy/internal/module
開啟Go Module后，go get增加了一個(gè)新的環(huán)境變量GOPROXY。該環(huán)境變量一旦開啟，go get就完全切換到新的取包流程，即GOPROXY流程。
在GOPROXY流程中，官方定義了一組代理接口, 請參考官方接口定義。
https://tip.golang.org/cmd/go/#hdr-Module_proxy_protocol
GET $GOPROXY/<module>/@v/list returns a list of all known versions of the given module, one per line.GET $GOPROXY/<module>/@v/<version>.info returns JSON-formatted metadata about that version of the given module.GET $GOPROXY/<module>/@v/<version>.mod returns the go.mod file for that version of the given module.GET $GOPROXY/<module>/@v/<version>.zip returns the zip archive for that version of the given module.
其實(shí)這組接口的定義就是$GOPATH/pkg/mod/cache/download中的文件系統(tǒng)。就是說，我們可以直接將此目錄下的文件系統(tǒng)作為代理使用，如下命令：
export GOPROXY=file:///$GOPATH/pkg/mod/cache/download/goproxy 其實(shí)很簡單，實(shí)現(xiàn)了上述四個(gè)接口的代理
% lsDockerfile contrib main.go scriptsLICENSE docker-compose.yaml proxy sumdbMakefile go.mod renameio testREADME.md go.sum robustio
先看下main.go文件
func main()handle = &logger{proxy.NewRouter(proxy.NewServer(new(ops)), &proxy.RouterOptions{Pattern: excludeHost,Proxy: proxyHost,DownloadRoot: downloadRoot,})}handle = &logger{proxy.NewServer(new(ops))}server := &http.Server{Addr: listen, Handler: handle}
注冊了一個(gè)ops server
ops實(shí)現(xiàn)了協(xié)議要求的接口
type ops struct{}func (*ops) List(ctx context.Context, mpath string) (proxy.File, error)func (*ops) Latest(ctx context.Context, path string) (proxy.File, error) {d, err := download(module.Version{Path: path, Version: "latest"})func (*ops) Info(ctx context.Context, m module.Version) (proxy.File, error)func (*ops) GoMod(ctx context.Context, m module.Version) (proxy.File, error)func (*ops) Zip(ctx context.Context, m module.Version) (proxy.File, error)
接著看下proxy/router.go文件
func NewRouter(srv *Server, opts *RouterOptions) *Routerrt := &Router{opts: opts,srv: srv,}remote, err := url.Parse(opts.Proxy)proxy := httputil.NewSingleHostReverseProxy(remote)proxy.Director = func(r *http.Request) {director(r)r.Host = remote.Host}rt.proxy.Transport = &http.Transport{Proxy: http.ProxyFromEnvironment,TLSClientConfig: &tls.Config{InsecureSkipVerify: true},}
調(diào)用了httputil的
httputil.NewSingleHostReverseProxy函數(shù)
func (rt *Router) ServeHTTP(w http.ResponseWriter, r *http.Request) {if strings.HasPrefix(r.URL.Path, "/sumdb/") {sumdb.Handler(mw, r)}if strings.HasSuffix(r.URL.Path, "/@latest") {}rt.proxy.ServeHTTP(mw, r)}func GlobsMatchPath(globs, target string) bool {matched, _ := path.Match(glob, prefix)}
最后看看proxy/server.go文件
首先注入ops
func NewServer(ops ServerOps) *Server {return &Server{ops: ops}}
然后ServeHTTP接口對ops的接口進(jìn)行了包裝和反向代理
func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {if strings.HasPrefix(r.URL.Path, "/sumdb/") {sumdb.Handler(w, r)}i := strings.Index(r.URL.Path, "/@")modPath, err := module.UnescapePath(strings.TrimPrefix(r.URL.Path[:i], "/"))switch what {case "latest":ctype = contentTypeJSONf, openErr = s.ops.Latest(ctx, modPath)case "v/list":ctype = contentTypeTextf, openErr = s.ops.List(ctx, modPath)default:what = strings.TrimPrefix(what, "v/")}switch ext {case ".info":ctype = "application/json"f, openErr = s.ops.Info(ctx, m)case ".mod":ctype = "text/plain; charset=UTF-8"f, openErr = s.ops.GoMod(ctx, m)case ".zip":ctype = "application/octet-stream"f, openErr = s.ops.Zip(ctx, m)default:http.Error(w, "request not recognized", http.StatusNotFound)return}http.ServeContent(w, r, what, info.ModTime(), f)
func ServeContent(w ResponseWriter, req *Request, name string, modtime time.Time, content io.ReadSeeker)該函數(shù)使用提供的ReaderSeeker提供的內(nèi)容來恢復(fù)請求,該函數(shù)相對于io.Copy的優(yōu)點(diǎn)是可以處理范圍類請求,設(shè)定MIME類型,并且處理了If-Modified-Since請求.如果未設(shè)定content-type類型,該函數(shù)首先通過文件擴(kuò)展名來判斷類型,如果失效的話,讀取content的第一塊數(shù)據(jù)并將他傳遞給DetectContentType進(jìn)行類型判斷.name可以不被使用,更進(jìn)一步說,他可以為空并且不在respone中返回.如果modtime不是0時(shí)間,該時(shí)間則體現(xiàn)在response的最后一次修改的header中,如果請求包括一個(gè)If-Modified-Since header,該函數(shù)利用modtime來決定是否發(fā)送該content.該函數(shù)利用Seek功能來決定content的大小.
推薦閱讀
評論
圖片
表情