Code Programming Mall Optimization Skills

SEO e-commerce Website Server Code Construction

Golang handles HTTP/HTTPS requests Parse the return result of JSON type

Posted on by led

I. Introduction

 HTTP cannot be bypassed. . . .
  1. Supplementary HTTP knowledge points
  2. The method of http operation HTTP defines different methods for interacting with the server. There are four basic methods, namely GET, POST, PUT, and DELETE. The full name of the URL is the resource descriptor. We can think of it this way: a URL address is used to describe a resource on the network, and GET, POST, PUT, and DELETE in HTTP correspond to the four operations of querying, modifying, adding, and deleting this resource. At this point, you should have a general understanding, GET is generally used to obtain/query resource information, and POST is generally used to update resource information. So, in addition to the four methods mentioned above, is there any other method for HTTP? In fact, HTTP defines the following request methods: GET method;
    POST method;
    PUT method;
    DELETE method.
    HEAD method;
    TRACE method;
    OPTIONS method;

1.Get is the most commonly used method, usually used to request the server to send a certain resource, and it should be safe and idempotent.

  (1). The so-called security means that the operation is used to obtain information rather than modify it. In other words, GET requests should generally not have side effects. That is to say, it only obtains resource information, just like a database query, it will not modify and add data, and will not affect the state of the resource.

Note: The meaning of security here only refers to non-modified information.

 (2). Idempotency means that multiple requests to the same URL should return the same result.
  1. The POST method submits data to the server, such as completing the submission of form data, and submitting the data to the server for processing.
  2. The PUT method asks the server to use the body of the request to create a new document named by the requested URL; if that document exists, replace it with the body.
  3. The DELETE method is to request the server to delete the resource corresponding to the specified URL. However, the client cannot guarantee that the delete operation will be performed, because the HTTP specification allows the server to withdraw the request without notifying the client.
  4. The HEAD method behaves similarly to the GET method, but the server returns only the body of the entity in the response. This allows the client to check the header of the resource without getting the actual resource,

Using HEAD, we can do the following more efficiently:
①. Understand some information about the resource, such as the resource type, without obtaining the resource;
②. By checking the status code in the response, you can determine whether the resource exists;
③. By looking at the header, test whether the resource has been modified.

  1. The TRACE method will initiate a “loopback” diagnosis on the destination server. We all know that when the client initiates a request, the request may go through a firewall, proxy, gateway, or some other application. Each node in between may modify the original HTTP request, and the TRACE method allows the client to finally send the request to the server, what it becomes. Due to a “loopback” diagnostic, when the request finally arrives at the server, the server will bounce a TRACE response with the final appearance of the original request message it received in the response body. In this way, the client can check whether the HTTP request message has been modified on the way.
  2. The OPTIONS method is used to obtain the methods supported by the current URL. If the request is successful, it will include a header named “Allow” in the HTTP header with the value of a supported method, such as “GET, POST”.
  1. http status code and meaning
    Status code Meaning
    1xx request is being processed
    2xx requests processed successfully
    3xx requests require additional actions, common examples are redirects
    4xx Client error caused the request to not be processed
    5xx Server processing error

GET request

1:Basic get request

package main
 
import (
    "fmt"
    "io/ioutil"
    "net/http"
)
 
func main() {
    resp, err := http.Get("http://httpbin.org/get")
    if err != nil {
        fmt.Println(err)
        return
    }
    defer resp.Body.Close()
    body, err := ioutil.ReadAll(resp.Body)
    fmt.Println(string(body))
    fmt.Println(resp.StatusCode)
    if resp.StatusCode == 200 {
        fmt.Println("ok")
    }
}

2:get request with parameters

package main
 
import (
    "fmt"
    "io/ioutil"
    "net/http"
)
 
func main(){
    resp, err := http.Get("http://httpbin.org/get?name=zhangsan&age=23")
    if err != nil {
        fmt.Println(err)
        return
    }
    defer resp.Body.Close()
    body, _ := ioutil.ReadAll(resp.Body)
    fmt.Println(string(body))
 
}

But what if we want to make some parameters into variables instead of putting them directly in the url, the code example is as follows:

package main
 
import (
     "fmt"
     "io/ioutil"
     "net/http"
     "net/url"
)
 
func main(){
     params := url.Values{}
     Url, err := url.Parse("http://httpbin.org/get")
     if err != nil {
         return
     }
     params.Set("name","zhangsan")
     params.Set("age","23")
     //If there are Chinese parameters in the parameters, this method will URLEncode
     Url.RawQuery = params.Encode()
     urlPath := Url.String()
     fmt.Println(urlPath) // https://httpbin.org/get?age=23&name=zhangsan
     resp,err := http.Get(urlPath)
     defer resp.Body.Close()
     body, _ := ioutil.ReadAll(resp.Body)
     fmt.Println(string(body))
}

Parse the return result of JSON type

package main
 
import (
    "encoding/json"
    "fmt"
    "io/ioutil"
    "net/http"
)
 
type result struct {
    Args string `json:"args"`
    Headers map[string]string `json:"headers"`
    Origin string `json:"origin"`
    Url string `json:"url"`
}
 
func main() {
    resp, err := http.Get("http://httpbin.org/get")
    if err != nil {
        return
    }
    defer resp.Body.Close()
    body, _ := ioutil.ReadAll(resp.Body)
    fmt.Println(string(body))
    var res result
    _ = json.Unmarshal(body,&res)
    fmt.Printf("%#v", res)
}

GET request add request header

package main
 
import (
    "fmt"
    "io/ioutil"
    "net/http"
)
 
func main() {
    client := &http.Client{}
    req,_ := http.NewRequest("GET","http://httpbin.org/get",nil)
    req.Header.Add("name","zhangsan")
    req.Header.Add("age","3")
    resp,_ := client.Do(req)
    body, _ := ioutil.ReadAll(resp.Body)
    fmt.Printf(string(body))
}

POST request

1. Basic post request
    package main
     
    import (
        "fmt"
        "io/ioutil"
        "net/http"
        "net/url"
    )
     
    func main() {
        urlValues ​​:= url.Values{}
        urlValues.Add("name","zhangsan")
        urlValues.Add("age","22")
        resp, _ := http.PostForm("http://httpbin.org/post",urlValues)
        body, _ := ioutil.ReadAll(resp.Body)
        fmt.Println(string(body))
    }
The result is as follows:
    {
      "args": {},
      "data": "",
      "files": {},
      "form": {
        "age": "22",
        "name": "zhangsan"
      },
      "headers": {
        "Accept-Encoding": "gzip",
        "Content-Length": "19",
        "Content-Type": "application/x-www-form-urlencoded",
        "Host": "httpbin.org",
        "User-Agent": "Go-http-client/1.1"
      },
      "json": null,
      "origin": "211.138.20.170, 211.138.20.170",
      "url": "https://httpbin.org/post"
    }
2. Another way
    package main
     
    import (
        "fmt"
        "io/ioutil"
        "net/http"
        "net/url"
        "strings"
    )
     
    func main() {
        urlValues ​​:= url.Values{
            "name":{"zhangsan"},
            "age":{"23"},
        }
        reqBody:= urlValues.Encode()
        resp, _ := http.Post("http://httpbin.org/post", "text/html",strings.NewReader(reqBody))
        body,_:= ioutil.ReadAll(resp.Body)
        fmt.Println(string(body))
    }
result
    {
      "args": {},
      "data": "age=23&name=zhangsan",
      "files": {},
      "form": {},
      "headers": {
        "Accept-Encoding": "gzip",
        "Content-Length": "19",
        "Content-Type": "text/html",
        "Host": "httpbin.org",
        "User-Agent": "Go-http-client/1.1"
      },
      "json": null,
      "origin": "211.138.20.170, 211.138.20.170",
      "url": "https://httpbin.org/post"
    }
3. Post request to send JSON data
    package main
     
    import (
        "bytes"
        "encoding/json"
        "fmt"
        "io/ioutil"
        "net/http"
    )
     
    func main() {
        client := &http.Client{}
        data := make(map[string]interface{})
        data["name"] = "zhangsan"
        data["age"] = "23"
        bytesData, _ := json.Marshal(data)
        req, _ := http.NewRequest("POST","http://httpbin.org/post",bytes.NewReader(bytesData))
        resp, _ := client.Do(req)
        body, _ := ioutil.ReadAll(resp.Body)
        fmt.Println(string(body))
     
    }
result
    {
      "args": {},
      "data": "{\"age\":\"23\",\"name\":\"zhangsan\"}",
      "files": {},
      "form": {},
      "headers": {
        "Accept-Encoding": "gzip",
        "Content-Length": "29",
        "Host": "httpbin.org",
        "User-Agent": "Go-http-client/1.1"
      },
      "json": {
        "age": "23",
        "name": "zhangsan"
      },
      "origin": "211.138.20.170, 211.138.20.170",
      "url": "https://httpbin.org/post"
    }
  1. HTTPS request The data transmitted by the HTTP protocol is unencrypted, that is, in plain text. Therefore, it is very insecure to use the HTTP protocol to transmit private information. In order to ensure that these private data can be encrypted and transmitted, Netscape designed the SSL (Secure Sockets Layer) protocol for HTTPS was born to encrypt the data transmitted by the HTTP protocol. In short, the HTTPS protocol is a network protocol constructed by the SSL+HTTP protocol that can perform encrypted transmission and identity authentication, and is more secure than the http protocol.

The main differences between HTTPS and HTTP are as follows:

  1. The https protocol needs to go to ca to apply for a certificate. Generally, there are few free certificates, so a certain fee is required.
  2. http is a hypertext transfer protocol, information is transmitted in plain text, and https is a secure ssl encrypted transfer protocol.
  3. http and https use completely different connection methods and different ports. The former is 80 and the latter is 443.
  4. The connection of http is very simple and stateless; the HTTPS protocol is a network protocol constructed by the SSL+HTTP protocol that can perform encrypted transmission and identity authentication, which is safer than the http protocol.

For https requests, we cannot get the desired data without a certificate. So the go code level has to be added to the certificate.

  1. Add the code for the certificate package xxxxxxxxxxxxxxxxxxxxxxxxxxx import (
    “crypto/tls”
    “crypto/x509”
    “io/ioutil”
    “net/http”
    “time”
    ) func GetHttps(url, caCertPath, certFile, keyFile string) ([]byte, error) { // Create certificate pool and various objects
    var pool *x509.CertPool // We need to store some certificates in this pool
    var client *http.Client
    var resp *http.Response
    var body []byte
    var err error var caCrt []byte // root certificate
    caCrt, err = ioutil.ReadFile(caCertPath)
    pool = x509.NewCertPool()
    if err != nil {
    return nil, err
    }
    pool.AppendCertsFromPEM(caCrt) var cliCrt tls.Certificate // specific certificate load object
    cliCrt, err = tls.LoadX509KeyPair(certFile, keyFile)
    if err != nil {
    return nil, err
    } // Pass the above prepared content to the client
    client = &http.Client{
    Transport: &http.Transport{
    TLSClientConfig: &tls.Config{
    RootCAs: pool,
    Certificates: []tls.Certificate{cliCrt},
    },
    },
    } // Get request
    resp, err = client.Get(url)
    if err != nil {
    return nil, err
    }
    defer resp.Body.Close() body, err = ioutil.ReadAll(resp.Body)
    if err != nil {
    return nil, err
    }
    defer client.CloseIdleConnections()
    return body, nil
    }

We take the certificate file from the server locally and pass it into the program as a parameter.

func TestGetHttps(t *testing.T) {
resp, err := GetHttps("https://xx.xx.xxx.xxx:xxxx/metrics",
"C:/Users/Desktop/ca.crt",
"C:/Users/Desktop/healthcheck-client.crt",
"C:/UsersDesktop/healthcheck-client.key")
if err != nil {
fmt.Println(err)
}
fmt.Println(string(resp))
}

The “crypto” standard library is used here, where we use

The tls.LoadX509KeyPair() method reads the certificate path and converts it to a certificate object;
The x509.NewCertPool() method creates a certificate pool;
The pool.AppendCertsFromPEM(caCrt) method adds the root certificate to the certificate pool.
  1. HTTPS request with token added in Header

After we have added the certificate, we still can’t get the data for some webpages that need to be authenticated. What should we do, add authentication.

func GetHttpsSkip(url, token string) ([]byte, error) {
// create various objects
var client *http.Client
var request *http.Request
var resp *http.Response
var body []byte
var err error
`Note here, use InsecureSkipVerify: true to skip certificate verification`
client = &http.Client{Transport: &http.Transport{
TLSClientConfig: &tls.Config{
InsecureSkipVerify: true,
},
}}
// get request request
request, err = http.NewRequest("GET", url, nil)
if err != nil {
log.Println("GetHttpSkip Request Error:", err)
return nil, nil
}
// add token
request.Header.Add("Authorization", token)
resp, err = client.Do(request)
if err != nil {
log.Println("GetHttpSkip Response Error:", err)
return nil, nil
}
defer resp.Body.Close()
body, err = ioutil.ReadAll(resp.Body)
defer client.CloseIdleConnections()
return body, nil
}

Pass in token, verify

func TestGetHttps(t *testing.T) {
resp, err := GetHttpsSkip("https://10.10.102.91:10250/metrics",
"Bearer eyxxxxxxxxxxxxxxxxxxxxx....xxxxx")
if err != nil {
fmt.Println(err)
}
fmt.Println(string(resp))
}

Leave a Reply

Your email address will not be published. Required fields are marked *

en_USEnglish
en_USEnglish