Forum Discussion

David_T's avatar
David_T
Icon for Nimbostratus rankNimbostratus
Nov 09, 2020

Modify XML POST request

Hi,

 

I need to change some values in an XML Post REQUEST to Sharepoint (search) as it embeds URL in the XML with https: and I need it to be http.

 

I was looking to use STREAM::EXPRESSION which I am successfully using for doing a similar replacement in the RESPONSE.

 

This is what I thought would work:

 

when HTTP_REQUEST {

  # Explicitly disable the stream profile for each client-side request so it doesn't stay

  #  enabled for subsequent HTTP requests on the same TCP connection.

  STREAM::disable

 

# The line below prevents the server from compressing its responses.

  HTTP::header remove "Accept-Encoding"

  # Apply stream profile against text responses from the application

if { [HTTP::method] eq "POST" } {

HTTP::collect

  set ispost 1

}

}

 

when HTTP_REQUEST_DATA {

  if { [HTTP::header value Content-Type] contains "text"} {

#replace https with http

STREAM::expression "@https://@http://@"

# enable the stream filter for this request only

STREAM::enable

}

//recalculate the content-length after the replacement

if { [info exists ispost] } {

HTTP::header insert Content-Length [HTTP::payload length]

}

}

 

But I am getting an HTTP 411 response from the backend (which I had thought recalculating the content-length header would fix).

 

Any pointers would be appreciated.

 

Regards,

David.

  • Don't use HTTP::header insert - then you'll get two Content-Length headers. Just check that there is already one present and update it. If it is not present then insert it

    if { [HTTP::header exists Content-Length] } {
      HTTP::header replace Content-Length $contentLength
    } else {
      HTTP::header insert Conent-Length $contentLength
    }
    • David_T's avatar
      David_T
      Icon for Nimbostratus rankNimbostratus

      Thanks Peter.

      After a lot of experimentation I came to the conclusion that using Stream for request processing just doesn't work - which may explain there are zilch examples I could find.

      Anyway I achieved what I wanted with the below:

      when HTTP_REQUEST {
         # Explicitly disable the stream profile for each client-side request so it doesn't stay
         #   enabled for subsequent HTTP requests on the same TCP connection.
         STREAM::disable
       
      	# The line below prevents the server from compressing its responses.
          HTTP::header remove "Accept-Encoding"
          # Apply stream profile against text responses from the application
          if { [HTTP::header value Content-Type] contains "text"} {
            HTTP::collect [HTTP::header Content-Length]
            set clen [HTTP::header Content-Length]      
          }
      }
       
      when HTTP_REQUEST_DATA {
          regsub -all "https://mysite" [HTTP::payload] "http://mysite" newdata
          HTTP::payload replace 0 $clen $newdata
          HTTP::release 
      }
       
      when HTTP_RESPONSE {
           # Explicitly disable the stream profile for each server-side response so it doesn't stay
         #   enabled for subsequent HTTP responses on the same TCP connection.
      STREAM::disable
        if { !([HTTP::header exists "Strict-Transport-Security"])} {
          HTTP::header insert "Strict-Transport-Security" "maxage=16070400; includeSubdomains"
        }
       
          # Apply stream profile against text responses from the application
          if { [HTTP::header value Content-Type] contains "text"} {
            # Look for the http:// and replace it with https://
            STREAM::expression {@http:@https:@ @//mysite/@//mysite.example/@}
            # Enable the stream profile for this response only
            STREAM::enable
         }
      }