Forum Discussion

Kevin_Basler's avatar
Kevin_Basler
Icon for Altostratus rankAltostratus
May 19, 2022

STREAM::expression doesn't seem to work

We've created an iRule that is supposed to change any occurance of the host internal.com to external.com in the payload of the response. We've added some logging steps to confirm that the STREAM::expression doesn't seem to be making the change.  Here's the code:

when HTTP_RESPONSE {

STREAM::disable

if { [HTTP::payload] contains "internal.com"} {
log local0. "Before if, payload contains : $[HTTP::payload]"
set oldString "internal.com"
set newString "external.com"
STREAM::expression @${oldString}@${newString}@@
STREAM::enable
log local0. "After if, payload contains : $[HTTP::payload]"
}
}

We've tried using the hosts internal.com and external.com in the stream expression instead of variables, several variations with brackets, quotes and nothing seem to make any difference. We do have the stream profile assigned to the VS.

Any suggestions would be appreciated. 

application delivery
  • Kevin_Basler's avatar
    Kevin_Basler
    May 24, 2022

    So this is the solution that we arrived at, thanks to you and your suggestion of using HTTP::header remove "Accept-Encoding". I've removed it and it fails just like before. Add it back in and everythign is working.

    when HTTP_REQUEST {
    HTTP::header remove "Accept-Encoding"
    }

    when HTTP_RESPONSE {
    if { [HTTP::header exists "Location"] }{
    set internal_host "https://internal.com"
    set external_host "https://external.com"
    if { [HTTP::header "Location"] contains $internal_host } {
    HTTP::header replace "Location" [string map "$internal_host $external_host" [HTTP::header "Location"]]
    }
    }
    STREAM::disable
    if { [HTTP::payload] contains "internal.com"} {
    log local0. "Before if, payload contains : $[HTTP::payload]"
    set oldString "internal.com"
    set newString "external.com"
    STREAM::expression @$oldString@$newString@@
    STREAM::enable
    log local0. "Before if, payload contains : $[HTTP::payload]"
    }
    }

    Something that is interesting is that I'm seeing readable payload both before and after using HTTP::header remove "Accept-Encoding" in the HTTP_REQUEST section. I can't explain it, but the important part is we have a working solution.

    Thanks for all of your help, Alex! Really appreciate it!

     

  • AlexBCT's avatar
    AlexBCT
    May 26, 2022

    Hi Kevin,

    Great to hear! Indeed interesting as to why this didn't work beforehand, but at least it's working now. Also thanks for posting to solution, I'm sure someone else will come across it and be happy with it as well. 😉 

  • AlexBCT's avatar
    AlexBCT
    Icon for Cumulonimbus rankCumulonimbus
    May 23, 2022

    Hi Kevin, 

    Do you have a compression profile attached to the virtual server? Most web servers will compress HTTP responses by default, which means that when the STREAM profile/irule looks through the return payload, it tries to match the compressed content, which doesn't contain the strings internal.com. 

    If you don't have it on the virtual servers, two things you can try; 

    1. attach a HTTP compression profile on your virtual server, this will offload HTTP compression onto the F5 (and saving a bit of CPU on the web server as well), or; 

    2. Using an iRule or Local Traffic Policy, remove the Accept-Encoding header from the request before sending it through to the web server. Be aware though that this will now serve up uncompressed pages to the client and may increase bandwidth usage. 

    I've done a quick test on my lab system and this does make your iRule work. 

    Hope this helps. 

    Edit: Here's some more information about the impact of compression on the stream profile: https://support.f5.com/csp/article/K39394712

    • Kevin_Basler's avatar
      Kevin_Basler
      Icon for Altostratus rankAltostratus
      May 23, 2022

      Hey Alex, appreciate the quick reply. I'm just trying to understand this. 

      If compression was the issue, would I still see the value of HTTP::payload both before and after the STEAM::expression? As I mentioned before, I'm currently seeing the value before and after the stream::expression has run. Unfortunately, it doesn't seem to be changed.

      Thanks,

      Kevin

      • AlexBCT's avatar
        AlexBCT
        Icon for Cumulonimbus rankCumulonimbus
        May 24, 2022

        Well, you would still see a payload, but wouldn't make any sense. (garbled text) If you did see readable text, then compression indeed wouldn't be the issue. 

        Can you tell some more about the payload that you are looking into? E.g. Is it HTML/JSON/JS/etc. and the amount of data per response? And just to confirm, the text "internal.com" is indeed in the payload of the HTTP response rather than the headers?