Forum Discussion

AdirZe's avatar
AdirZe
Icon for Altocumulus rankAltocumulus
Feb 01, 2023

Using string map or regsub breaks non-english (UTF-8) characters

Hi Dear community ,

I woud appreciate your input. I want to replace a string inside a HTTP::payload with string map or regsub and then HTTP::payload replace. The replacement does take place but it also break the non-english [UTF-8] characters inside the HTML.

I tried implementing all the soultions I found inside DevCentral. Including these:

https://community.f5.com/t5/technical-forum/http-collect-results-in-garbled-http-payload-when-using-non/td-p/266407

https://support.f5.com/csp/article/K22406348

https://clouddocs.f5.com/api/irules/HTTP__payload.html

  • Note that the argument will be interpreted as a byte array. If it is actually a UTF-8 string with multibyte characters, the output will not be what you expect. In order to prepare a UTF-8 string for use as input to HTTP::payload replace, you should first run ‘binary scan c* throwawayvariable’.

 

Btw, Stream profile does work fine , but it's not very compatible with APM Virtual Servers.

Best Regards,

Adir

 

 

application delivery
  • AdirZe's avatar
    AdirZe
    Icon for Altocumulus rankAltocumulus
    Feb 08, 2023

    @G-Rob Hi Rob,

    It's very weird becasue I already tried this command before I open this thread - You can see I event reference this K article in my original post. But Thanks to you I did try it again, and it looks like now it's working great, Thanks a ton. 

    Best Regards,

    Adir

  • G-Rob's avatar
    G-Rob
    Icon for Employee rankEmployee
    Feb 02, 2023

    Adir,

    Thanks for posting your question on DevCentral. Could you post your current iRule for easier diagnosis?

    Thanks

    • AdirZe's avatar
      AdirZe
      Icon for Altocumulus rankAltocumulus
      Feb 04, 2023

      G-Rob 

      when RULE_INIT {
      # Limit payload collection to 5Mb
      set static::max_collect_len 5368709
      # Max characters to log locally (must be less than 1024 bytes)
      set static::max_chars 900
      }

      when HTTP_REQUEST {
      set collect 0
      if { [HTTP::path] eq "*" } {
      set collect 1
      }
      }

      when HTTP_RESPONSE {
      if { $collect eq 1 } {
      if {[HTTP::header exists "Content-Length"]}{
      set collect_length_resp [HTTP::header "Content-Length"]
      log local0. "Content-Length exists [HTTP::header "Content-Length"]"
      } else {
      HTTP::payload rechunk
      set collect_length_resp $static::max_collect_len
      }
      HTTP::collect $collect_length_resp
      }
      }


      when HTTP_RESPONSE_DATA {
      log local0. "Response Collected [HTTP::payload length] bytes"
      set newData [string map { * **} [HTTP::payload]]
      HTTP::payload replace 0 $collect_length_resp $newData
      HTTP::payload rechunk
      HTTP::release
      }

       

      Edited by Leslie_Hubertus to tag G-Rob to make sure he sees the follow-up. 🙂