Forum Discussion

Tuckson's avatar
Tuckson
Icon for Altostratus rankAltostratus
Mar 05, 2021

Simple reversed http check

Hi,

 

As first, I am aware similar questions have been asked often and for years now. However, somehow the answers I found tusfar don't satisfy me, so I'll try my own question :-)

 

We run hundreds of servers in big numbers of pools on our F5. Our networking department disallows usage of api's, however they want to coöperate in the following:

 

I need a a way to enable and disable poolmembers / nodes (will call all nodes from here) from a script or pipeline.

So I set up a webserver and set up this idea:

I define a healthcheck template that checks a page which is the nodename. If page is available, a status 200 is returned which is a sign to disable the node. When the page is absent, a 404 status is returned, which is a sign to enable the node. If anything else is returned also the node should be enabled.

 

Now I would like to know if the nodename is available a as default variable for a monitor?

 

If not, I need to define specific for each member, that is an awful lot of work. Would prefer to keep it general.

 

Also, can I achieve this without using an external monitor? Since there are 100's of members, the extra resource penalty might add up.

 

Thnx

  • > We run hundreds of servers in big numbers of pools on our F5. Our networking department disallows usage of api's, however they want to coöperate in the following:

     

    Using a script and the iControl RestAPI is by far the easiest way to control node availability.

     

    > I define a healthcheck template that checks a page which is the nodename.

     

    You mean you have a single node that provides an appropriate response as to whether a specific node should be available or not?

    You would have to do that using a custom external monitor.

     

    If every node had a page (the same page on all nodes) that reported whether it should be enabled or disabled then it might be easier.

     

    K71282813: Overview of BIG-IP EAV external monitors

     

    > Now I would like to know if the nodename is available a as default variable for a monitor?

     

    The NODE_NAME is set as an environmental variable for external monitors

     

    > Also, can I achieve this without using an external monitor? Since there are 100's of members, the extra resource penalty might add up.

     

    Not the way you describe it, and yes - you would be running the external monitor for all nodes, which would be a real performance drain.

     

    If you had a standard http monitor that probed every node to flag whether is should be enabled or disabled, you could use the log messages (via alertd) to trigger an icall script that enables/disables the node.

     

     

     

    • Tuckson's avatar
      Tuckson
      Icon for Altostratus rankAltostratus

      ===

      > You mean you have a single node that provides an appropriate response as to whether a specific node should be available or not? You would have to do that using a custom external monitor.

      ===

      The whole idea relies on the use of healthchecks on the F5. There are different forms of health checks. The most basic one is just checking if a listener port is available. If not, node disabled. If so, node enabled.

      However a more advanced check is also possible in which a send string is defined en a receive string is captured which can be used to enable or disable a node. We want to make use of this feature.

      The idea is that every node in an F5 vip has 2 diffferent healthchecks.

       

      1) The regular Port availability check.

      Is the port down then the node must be disabled. If the port comes up, the node should be enabled after a little while.

      Suppose we have a VIP1, which contains 2 nodes, ServerA and ServerB (see picture below).

      Normally, ServerA and ServerB are running and the VIP is up. Whenever 1 of these 2 servers comes in trouble and the port is no longer available, the port availability check sees so and disables the node in VIP1.

       

      2) The http check.

      Inside the environment we run a webserver (listens on port 8000). Let's call it ServerX. This webserver bij default returns a 404 status (page not found) on every request

      When we as engineers intentionally want to disable a node, we will let the webserver respond a status 200 to a request with the node's name.

       

      So per example:

       

      ServerA listening on port 8080 is placed in VIP1 on the F5 with 2 healtchecks. The port availability check and the http check. So is ServerB

      Now the port availabilty check just checks ServerA:8080

      The httpcheck however, sends it's request to http://serverX:8000/ServerA

       

      If a status 200 is returned, the F5 DISABLES ServerA in VIP1.  If ANYTHING ELSE is returned the F5 ENABLES ServerA in VIP1.

       

      For ServerD in VIP2, the same principle goes. Suppose the server listens on port 51243

      Now the port availabilty check just checks ServerD:51243

      The httpcheck however, sends it's request to http://serverX:8000/ServerD

       

      If a status 200 is returned, the F5 DISABLES ServerD in VIP2.  If ANYTHING ELSE is returned the F5 ENABLES ServerD in VIP2.

       

       

      The idea behind this reversed check (enabling on an error status) is that failures involving the webserver itselves should not lead to disabling nodes.

       

      Now, by being able to pass the nodename as a variable, there is no need to define the check specifically for each node.

       

       

      Ow, and I do not have acces to the restapi

      • Simon_Blakely's avatar
        Simon_Blakely
        Icon for Employee rankEmployee

        So ServerX:8000 is the node availability check - you probe serverX:8000 with the node-name to see if the node should be enabled or disabled.

         

        You would need a single external monitor applied to all the nodes that gets the NODE_NAME from the execution environment, and makes the probe, and then marks the node down based on the response code.

         

        Doable, but possibly a resource drag.

        I can't see a way to use a standard monitor to do this.

  • OK. Thnx. I was hoping that maybe a scripted monitor would be a possibility. But I read that Node_Name is available in external montors only anyway?

     

  • OK, It's working with an external monitor now. Nice to know the principle works. However I'm still a bit worried about possible performance impact when using this on al larger scale. SO am looking for ways to use , for example, a scripted monitor. Soms some form of expect, so this could be an alternative. Do I have ip address of the node/poolmember available in there?

     

    • Simon_Blakely's avatar
      Simon_Blakely
      Icon for Employee rankEmployee

      > SO am looking for ways to use , for example, a scripted monitor. Soms some form of expect, so this could be an alternative. Do I have ip address of the node/poolmember available in there?

       

      I have been working in Support for 7 years, and have never looked at scripted monitors.

      Nothing I have seen leads me to believe that it supports anything more than send/expect commands - ie they do not support the full range of TCL scripting available in expect.

       

      > I'm still a bit worried about possible performance impact when using this on al larger scale.

       

      All you can do is scale it up and monitor - keep the check intervals large.

  • Simon, thnk you for taking the time to answer my beginner questions ;-) Appreciate.

     

    OK. Another question on this then. Am tying to explore every possibility possible. So I bumped into the iRules and am reading on these now. As I understand they are trigggered on (for example) send and receive commands. So If I have ip:port of node-name/poolmember name available in these iRules I could perform an inline replacement in the url before sending.

     

    Is that assumption correct? If so, the 100mln dollar question: do I have the address:port or the node/member name available in the iRules?

    • Simon_Blakely's avatar
      Simon_Blakely
      Icon for Employee rankEmployee

      Irule triggers exist in the context of connections through a virtual server. Irules do not trigger for monitors which are managed by an external (to traffic processing) process called bigd.