/*****************************************************************************/ /* HTTP2.c HTTP/2 is a replacement for how HTTP is expressed "on the wire". It is not a ground-up rewrite of the protocol; HTTP methods, status codes and semantics are the same, and it should be possible to use the same APIs as HTTP/1.x (possibly with some small additions) to represent the protocol. The focus of the protocol is on performance; specifically, end-user perceived latency, network and server resource usage. One major goal is to allow the use of a single connection from browsers to a Web site. Some useful (and used) sites: https://tools.ietf.org/html/rfc9113 https://tools.ietf.org/html/rfc7541 https://en.wikipedia.org/wiki/HTTP/2 https://http2.github.io/ https://http2.github.io/faq/ http://http2-explained.haxx.se/ http://chimera.labs.oreilly.com/books/1230000000545/ch12.html https://insouciant.org/tech/http-slash-2-considerations-and-tradeoffs/ http://undertow.io/blog/2015/04/27/An-in-depth-overview-of-HTTP2.html https://blog.newrelic.com/2016/02/17/http2-production/ https://blog.newrelic.com/2016/02/09/http2-best-practices-web-performance/ https://blog.cloudflare.com/tools-for-debugging-testing-and-using-http-2/ https://nghttp2.org/ https://nghttp2.org/documentation/nghttp.1.html https://nghttp2.org/documentation/h2load.1.html https://nghttp2.org/blog/2014/11/29/test-your-http-slash-2-server-with-nghttp-client/ chrome://net-internals#http2 HTTP/2 PROCESSING ----------------- These are indicative processing flows only. Not all functions involved or encountered are shown. Too many if-buts-maybees. Use WATCH. Request Processing ~~~~~~~~~~~~~~~~~~ : | RequestGet() !match the HTTP/2 request preface | Http2Preface() !either of these two functions are Http2SwitchResponse() !entry points to HTTP/2 processing | Http2ConnectGet() !create HTTP/2 structure and dispose | !of initiating request structure | Http2ClientRead() <--+ <--+ !receive frame from client | | | | {control frames} | !HTTP/2 internal processing | | : +--{not header frames}--+ : !frame processed by type | . HpackHeadersFrame() !request headers frame | Http2RequestBegin() !initialise request processing | Http2RequestProcess() !begin processing the request | RequestParseDictionary() !dictionary contains request | : !as per any other request | Http2RequestEnd() !end the request HTTP/2 specifics | : Read Frame Processing ~~~~~~~~~~~~~~~~~~~~~ : Http2NetClientRead() !initiate HTTP/2 connection read loop | NetIoRead() !asynchronous read | : | Http2NetClientReadAst() <-- + !asynchronous read completion | | !validate frame : | : !switch/case on frame type : !call function to process frame | : NetIoRead() : !asynchronous read | | +----------------------------+ Write Frame Processing ~~~~~~~~~~~~~~~~~~~~~~ : NetIoWrite() !request standard network call | Http2NetIoWrite() !HTTP/2 write | Http2NetQueueWrite() <----- + !put the write on a queue | | +-->!add I/O to a queue --> : !return after putting in queue | : !check queue(s) for I/O : !no queued I/O then end processing | : NetIoWrite() >------------+ : : : Http2NetWriteHeaderAst() <-+ : !write frame header then data | : : NetIoWrite() >------------+ : : : Http2NetWriteDataAst() <--+ : !write frame header plus data | | NetIoWrite() >---------------+ WATCHing via HTTP/2 ------------------- If a WATCHing request and one instantiated by Http2RequestBegin() share the HTTP/2 connection then none of the HTTP/2 WATCH points will be reported because down that particular rabbit hole is found only madness. HTTP/2 DoS Mitigations ---------------------- https://tools.ietf.org/html/rfc7540#section-10.5 https://www.kb.cert.org/vuls/id/605641/ https://github.com/Netflix/security-bulletins/blob/master/advisories/third-party/2019-002.md WASD places explicit (and reasoanble) limits on some HTTP/2 behaviours. Though exercising these mitigations is beyond the scope of the WASD test-bench. When mitigations activate the connection is dropped. Fingers crossed. 1) CVE-2019-9511, also known as Data Dribble WASD's asynchronous buffering and lack of stream prioritising mitigates. 2) CVE-2019-9512, also known as Ping Flood Http2Ping() checks for unreasonable client ping requests per second. 3) CVE-2019-9513, also known as Resource Loop Stream prioritising is not implemented by WASD. 4) CVE-2019-9514, also known as Reset Flood Http2StreamReset() checks for unreasonable server stream resets per second. 5) CVE-2019-9515, also known as Settings Flood Http2Settings() checks for unreasonable client setting frames per second. 6) CVE-2019-9516, also known as 0-Length Headers Leak HpackHeadersFrame() checks for zero-sized client headers. 7) CVE-2019-9517, also known as Internal Data Buffering The use of the TCP sliding window in this DoS puts it beyond WASD's control. WASD's asynchronous I/O should limit buffering impacts. 8) CVE-2019-9518, also known as Empty Frame Flooding Http2NetClientReadAst() checks for unreasonable client frame sizes. HTTP/2 Flow Control ------------------- https://nghttp2.org/blog/2014/11/29/test-your-http-slash-2-server-with-nghttp-client/ https://medium.com/coderscorner/http-2-flow-control-77e54f7fd518 https://stackoverflow.com/questions/34722343/flow-control-questions-about-http-2-rfc-implementation https://ably.com/topic/http2 https://github.com/square/okhttp/issues/6749 TESTING ------- Needless to say the major browsers (Chrome, Edge, FireFox, Safari) all contributed to end-use development. Indispensible were the |nghttp| and the associated |h2load| tools running on a Linux Mint (17.3) VM. Many thanks to the developer(s) of this package. And of course for someone educated in computing during the (19)70s, the availability of VM technology for such purposes is just brilliant! "But you know, we were happy in those days, though we were poor." o exercise the basic HTTP/2 functionality nghttp -nv https://klaatu.private/ o exercise starting HTTP/2 for "http" URIs (RFC 7450 3.2) nghttp -nvu http://klaatu.private/ o exercise starting HTTP/2 (for "http") with prior knowledge (RFC 7450 3.4) nghttp -nv http://klaatu.private/ o headers with continuation frames (RFC 7450 6.10) nghttp --continuation https://klaatu.private/ o basic HTTP/2 loading (number, clients and streams bumped) h2load --requests=1000 --clients=5 --max-concurrent-streams=5 https://klaatu.private/ o HTTP/2 loading with defined URIs h2load --requests=1000 --clients=5 --threads=5 --max-concurrent-streams=5 --input-file=urls.txt o HTTP/1.1 loading with defined URIs h2load --h1 --requests=100 --clients=2 --threads=2 --max-concurrent-streams=5 --input-file=urls.txt o exercise flow control nghttp -nv -W17 -w15 https://wasd.kicks-ass.net/stream/text:1000 nghttp -nv -W14 -w12 https://wasd.kicks-ass.net/stream/text:1000 nghttp -nv -W14 -w10 https://wasd.kicks-ass.net/stream/text:1000 The HTTP/2 and HTTP/1.1 with --input-file can be concurrently used in two separate sessions to simultaneously exercise the server against the two protocols. The -w17 -W17 (for example) switches can exercise flow control. When module WATCHing is compiled-in, defining the WASD_HTTP2_SUBTLE_BREAK logical name enables code to break request processing in specific but subtle ways to ensure the server's continued stability in the presence of network or client issues. It should be used under load from soemthing akin to |h2load|. Testing specific errors; using https://github.com/summerwind/h2spec FOR LOCAL REFERENCE ------------------- +-----------------------------------------------+ | Length (24) | +---------------+---------------+---------------+ | Type (8) | Flags (8) | +-+-------------+---------------+-------------------------------+ |R| Stream Identifier (31) | +=+=============================================================+ | Frame Payload (0...) ... +---------------------------------------------------------------+ VERSION HISTORY --------------- 04-MAR-2024 MGD refine HTTP/2 flow control /DO=ZERO=HTTP2 via Http2ZeroAccounting() 27-OCT-2023 MGD HTTP/2 refinements using https://github.com/summerwind/h2spec 22-OCT-2023 MGD Http2Supervisor() mitigate Rapid Reset CVE-2023-44487 28-SEP-2022 MGD Http2ConnectClose() close stream requests 24-MAY-2022 MGD use ->TcpMaxQio to set initial frame size 15-MAR-2022 MGD Http2Supervisor() kludge to detect stalled processing 25-AUG-2021 MGD Http2ConnectClose() mung active flow control bugfix; Http2Supervisor() idle connection 03-APR-2021 MGD bugfix; Http2ConnectClose() once aborted always aborted 17-AUG-2020 MGD HTTP2_DEFAULT_WINDOW_SIZE from 1048575 to 131070 05-APR-2020 MGD Http2Report() remove proxy requests from HTTP/2-HTTP/1 ratio 31-JAN-2020 MGD fiddle with the code (perhaps optimisations, even bugfixes) 13-JAN-2019 MGD bugfix; Http2Priority() exclusive bit 17-NOV-2019 MGD bugfix; Http2ConnectClose() return after RequestEnd()ing 12-OCT-2019 MGD mitigate potential HTTP/2 DoS vulnerabilities https://www.kb.cert.org/vuls/id/605641/ https://tools.ietf.org/html/rfc7540#section-10.5 18-APR-2018 MGD bugfix; Http2StreamReset() rundown the request in-line 07-APR-2018 MGD bugfix; Http2StreamReset() explicitly rundown the request 23-DEC-2017 MGD bugfix; window update and flow control management 02-AUG-2015 MGD initial */ /*****************************************************************************/ #ifdef WASD_VMS_V7 # undef __VMS_VER # define __VMS_VER 70000000 # undef __CRTL_VER # define __CRTL_VER 70000000 #else # ifdef WASD_VMS_V7 # undef _VMS__V6__SOURCE # define _VMS__V6__SOURCE # undef __VMS_VER # define __VMS_VER 70000000 # undef __CRTL_VER # define __CRTL_VER 70000000 # endif #endif #include #include #include "wasd.h" #define WASD_MODULE "HTTP2" /******************/ /* global storage */ /******************/ BOOL Http2Enabled; int Http2CurrentConnected, Http2CurrentProcessing, Http2IdleSeconds, Http2PingSeconds; uint Http2InitialWindowSize, Http2MaxConcurrentStreams, Http2MaxFrameSize, Http2MaxHeaderListSize, Http2MaxHeaderTableSize, Http2StreamIdent; /* used by VM.c */ const int Http2StructSize = sizeof(HTTP2_STRUCT); char Http2ClientPreface [] = "PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n"; int Http2ClientPrefaceLength = sizeof(Http2ClientPreface)-1; char *Http2ErrorArray [HTTP2_ERROR_COUNT] = { "graceful shutdown", "protocol error detected", "implementation fault", "flow-control limits exceeded", "settings not acknowledged", "frame received for closed stream", "frame size incorrect", "stream not processed", "stream cancelled", "compression state not updated", "TCP connection error for CONNECT method", "processing capacity exceeded", "negotiated TLS parameters not acceptable", "use HTTP/1.1 for the request" }; LIST_HEAD Http2List; /********************/ /* external storage */ /********************/ extern int HttpdSizeOfHttp2, HttpdSizeOfStream2, InstanceNodeConfig, InstanceNodeSupervisor, NetReadBufferSize; extern ulong HttpdTickSecond; extern char ErrorSanityCheck []; #define acptr AccountingPtr extern ACCOUNTING_STRUCT *AccountingPtr; extern CONFIG_STRUCT Config; extern LIST_HEAD RequestList; extern PROXY_ACCOUNTING_STRUCT *ProxyAccountingPtr; extern WATCH_STRUCT Watch; /*****************************************************************************/ /* Initialise all things HTTP/2. */ void Http2Init () { /*********/ /* begin */ /*********/ if (WATCH_MODULE(WATCH_MOD_HTTP2)) WatchThis (WATCHALL, WATCH_MOD_HTTP2, "Http2Init()"); /* won't detect member shuffles but will pickup modifications to size */ if (HttpdSizeOfHttp2 != sizeof(HTTP2_STRUCT) || HttpdSizeOfStream2 != sizeof(HTTP2_STREAM_STRUCT)) ErrorExitVmsStatus (SS$_BUGCHECK, "HTTP/2 structure(s) mismatch", FI_LI); Http2Enabled = Config.cfHttp2.Enabled; FaoToStdout ("%HTTPD-I-HTTP2, !AZ\n", Http2Enabled ? "enabled" : "disabled"); Http2InitialWindowSize = Config.cfHttp2.InitialWindowSize; if (!Http2InitialWindowSize) Http2InitialWindowSize = HTTP2_DEFAULT_WINDOW_SIZE; else if (Http2InitialWindowSize < HTTP2_INITIAL_WINDOW_SIZE) Http2InitialWindowSize = HTTP2_INITIAL_WINDOW_SIZE; else if (Http2InitialWindowSize > HTTP2_MAX_WINDOW_SIZE) Http2InitialWindowSize = HTTP2_MAX_WINDOW_SIZE; Http2MaxConcurrentStreams = Config.cfHttp2.MaxConcurrentStreams; if (!Http2MaxConcurrentStreams) Http2MaxConcurrentStreams = HTTP2_INITIAL_MAX_CONC_STREAMS; if (Http2MaxConcurrentStreams < HTTP2_MIN_INITIAL_MAX_CONC_STREAMS) Http2MaxConcurrentStreams = HTTP2_MIN_INITIAL_MAX_CONC_STREAMS; Http2MaxFrameSize = Config.cfHttp2.MaxFrameSize; if (Http2MaxFrameSize < HTTP2_INITIAL_MAX_FRAME_SIZE) Http2MaxFrameSize = HTTP2_INITIAL_MAX_FRAME_SIZE; else if (Http2MaxFrameSize > HTTP2_MAX_FRAME_SIZE) Http2MaxFrameSize = HTTP2_MAX_FRAME_SIZE; if (Http2Enabled && NetReadBufferSize < Http2MaxFrameSize) { NetReadBufferSize = Http2MaxFrameSize; FaoToStdout ("%HTTPD-W-HTTP2, network read buffer size \ increased to !UL bytes\n", NetReadBufferSize); } if (!(Http2IdleSeconds = Config.cfTimeout.Http2Idle)) Http2IdleSeconds = HTTP2_TIMEOUT_IDLE_SECONDS; Http2MaxHeaderListSize = Config.cfHttp2.MaxHeaderListSize; if (Http2MaxHeaderListSize < HTTP2_INITIAL_HEAD_LIST_SIZE) Http2MaxHeaderListSize = HTTP2_INITIAL_HEAD_LIST_SIZE; else if (Http2MaxHeaderListSize > HTTP2_MAX_HEAD_LIST_SIZE) Http2MaxHeaderListSize = HTTP2_MAX_HEAD_LIST_SIZE; Http2MaxHeaderTableSize = Config.cfHttp2.MaxHeaderTableSize; if (Http2MaxHeaderTableSize < HTTP2_INITIAL_HEAD_TAB_SIZE) Http2MaxHeaderTableSize = HTTP2_INITIAL_HEAD_TAB_SIZE; else if (Http2MaxHeaderTableSize > HTTP2_MAX_HEAD_TAB_SIZE) Http2MaxHeaderTableSize = HTTP2_MAX_HEAD_TAB_SIZE; Http2PingSeconds = Config.cfHttp2.PingSeconds; if (Http2PingSeconds == 0) Http2PingSeconds = HTTP2_PING_SECONDS_DEFAULT; else if (Http2PingSeconds < 0) Http2PingSeconds = 0; /* "NONE" */ else if (Http2PingSeconds > HTTP2_PING_SECONDS_MAX) Http2PingSeconds = HTTP2_PING_SECONDS_MAX; if (!Http2Enabled) return; VmHttp2Init (); } /*****************************************************************************/ /* Called by WatchSetWatch(). */ void Http2SetWatch ( HTTP2_STRUCT *h2ptr, int item ) { h2ptr->WatchItem = item; if (h2ptr->NetIoPtr) h2ptr->NetIoPtr->WatchItem = item; } /*****************************************************************************/ /* This is a TLS/SSL (https:) encrypted connection that has specified HTTP/2 via the TLS ALPN negotiation and transmitted the HTTP/2 client connection preface (RFC 7540 3.3) --OR-- a clear-text (http:) connection prior-knowledge (RFC 7540 3.4) connection preface. Create an HTTP/2 structure and then dispose of the request structure used to initiate the network connection. */ BOOL Http2Preface (REQUEST_STRUCT *rqptr) { int count, idx; HTTP2_STRUCT *h2ptr; /*********/ /* begin */ /*********/ if (WATCHMOD (rqptr, WATCH_MOD_HTTP2)) WatchThis (WATCHITM(rqptr), WATCH_MOD_HTTP2, "Http2Preface() preface:!UL read:!UL http2:!UL", Http2ClientPrefaceLength, rqptr->NetIoPtr->ReadCount, rqptr->NetIoPtr->ReadCount - Http2ClientPrefaceLength); if (WATCHING (rqptr, WATCH_CONNECT)) WatchThis (WATCHITM(rqptr), WATCH_CONNECT, "HTTP/2 connection preface (h2)"); if (!Http2Enabled) return (false); if (WATCHING (rqptr, WATCH_CONNECT)) WatchThis (WATCHITM(rqptr), WATCH_CONNECT, "HTTP/2 protocol"); h2ptr = Http2ConnectGet (rqptr); /* if HTTP/2 traffic has been pipelined with the preface */ if ((count = rqptr->NetIoPtr->ReadCount - Http2ClientPrefaceLength) > 0) { if (count > h2ptr->ReadBufferSize) ErrorExitVmsStatus (SS$_BUGCHECK, ErrorSanityCheck, FI_LI); memcpy (h2ptr->ReadBufferPtr, rqptr->rqNet.ReadBufferPtr + Http2ClientPrefaceLength, count); h2ptr->NetIoPtr->ReadStatus = SS$_NORMAL; h2ptr->NetIoPtr->ReadCount = count; h2ptr->ReadInUseCount++; SysDclAst (Http2NetClientReadAst, h2ptr); } else SysDclAst (Http2NetClientRead, h2ptr); /* remove the seconded I/O structure by Http2ConnectGet() */ rqptr->NetIoPtr = NULL; RequestEnd5 (rqptr); InstanceMutexLock (INSTANCE_MUTEX_HTTPD); NetUpdateConnected (rqptr, -1); NetUpdateConnected (h2ptr, +1); InstanceMutexUnLock (INSTANCE_MUTEX_HTTPD); return (true); } /****************************************************************************/ /* This is an unencrypted connection (http:) with request header indicating a desire to upgrade to HTTP/2 (RFC 7540 3.2). Major browser communities indicate no interest in providing HTTP/2 over clear-text connections so it's a bit of a niche but supported by the RFC. Generate an HTTP/2 101 Switching Protocols response and then handle the original request in the HTTP/2 environment. Exercise using nghttp -nvu http:/// */ BOOL Http2SwitchResponse (REQUEST_STRUCT *rqptr) { static char Http2Switching [] = "HTTP/1.1 101 Switching Protocols\r\n\ Upgrade: h2c\r\n\ Connection: Upgrade\r\n\ \r\n"; int retval, SettingsLength; char *cptr; uchar SettingsBuffer [256]; HTTP2_STRUCT *h2ptr; /*********/ /* begin */ /*********/ if (WATCHMOD (rqptr, WATCH_MOD_RESPONSE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_RESPONSE, "Http2SwitchResponse()"); if (WATCHING (rqptr, WATCH_CONNECT)) WatchThis (WATCHITM(rqptr), WATCH_CONNECT, "HTTP/2 upgrade"); if (!Http2Enabled) return (false); if (rqptr->rqHeader.UpgradeHttp2onHttp && rqptr->ServicePtr->RequestScheme != SCHEME_HTTP) { rqptr->rqResponse.HttpStatus = 400; ErrorGeneral (rqptr, MsgFor(rqptr,MSG_REQUEST_FORMAT), FI_LI); RequestEnd (rqptr); return (true); } if (rqptr->rqHeader.Http2SettingsPtr != NULL) { /* span the length of base-64 acceptable characters */ for (cptr = rqptr->rqHeader.Http2SettingsPtr; isalnum(*cptr) || *cptr == '+' || *cptr == '/' || *cptr == '='; *cptr++); SettingsLength = sizeof(SettingsBuffer); retval = base64_decode (SettingsBuffer, &SettingsLength, rqptr->rqHeader.Http2SettingsPtr, cptr - rqptr->rqHeader.Http2SettingsPtr); } if (rqptr->rqHeader.Http2SettingsPtr == NULL || retval != 0) { rqptr->rqResponse.HttpStatus = 400; ErrorGeneral (rqptr, MsgFor(rqptr,MSG_REQUEST_FORMAT), FI_LI); RequestEnd (rqptr); return (true); } /****************/ /* 101 response */ /****************/ /* no update to status code counters as this is an intermediate response */ rqptr->rqResponse.HttpStatus = 101; rqptr->rqResponse.HeaderSent = true; /* blocking write (for the convenience) */ NetWrite (rqptr, NULL, Http2Switching, sizeof(Http2Switching)-1); /******************/ /* HTTP/2 request */ /******************/ if (WATCHING (rqptr, WATCH_CONNECT)) WatchThis (WATCHITM(rqptr), WATCH_CONNECT, "HTTP/2 protocol"); h2ptr = Http2ConnectGet (rqptr); /* remove the seconded I/O structure by Http2ConnectGet() */ rqptr->NetIoPtr = NULL; h2ptr->ExpectingH2cPreface = true; SysDclAst (Http2NetClientRead, h2ptr); InstanceMutexLock (INSTANCE_MUTEX_HTTPD); NetUpdateConnected (rqptr, -1); NetUpdateConnected (h2ptr, +1); InstanceMutexUnLock (INSTANCE_MUTEX_HTTPD); RequestEnd5 (rqptr); return (true); } /*****************************************************************************/ /* This is where the real fun begins. Create and initialise an HTTP/2 structure, using some data from the request structure. Return a pointer to the structure. */ HTTP2_STRUCT* Http2ConnectGet (REQUEST_STRUCT *rqptr) { HTTP2_STRUCT *h2ptr; /*********/ /* begin */ /*********/ if (WATCHMOD (rqptr, WATCH_MOD_HTTP2)) WatchThis (WATCHITM(rqptr), WATCH_MOD_HTTP2, "Http2ConnectGet()"); h2ptr = VmHttp2Get (rqptr->ConnectNumber); /* add to the HTTP/2 connection list */ ListAddHead (&Http2List, h2ptr, LIST_ENTRY_TYPE_HTTP2); if (rqptr->WatchItem) h2ptr->WatchItem = WatchSetWatch (NULL, WATCH_NEW_ITEM); if (!rqptr->NetIoPtr->TcpMaxQio) { TcpIpSocketSndBuf (rqptr->NetIoPtr); TcpIpSocketRcvBuf (rqptr->NetIoPtr); } /* second the request I/O structure to the HTTP/2 structure */ h2ptr->NetIoPtr = rqptr->NetIoPtr; h2ptr->NetIoPtr->WatchItem = h2ptr->WatchItem; h2ptr->ConnectTime64 = rqptr->rqTime.BeginTime64; /* always use original client data (see MapUrl_SetClientAddress()) */ if (rqptr->ClientResetPtr) h2ptr->ClientPtr = rqptr->ClientResetPtr; else h2ptr->ClientPtr = rqptr->ClientPtr; /* use the same service of the upgrade request */ h2ptr->ServicePtr = rqptr->ServicePtr; /* intialise server protocol parameters */ h2ptr->ServerInitialWindowSize = Http2InitialWindowSize; h2ptr->ServerMaxConcStreams = Http2MaxConcurrentStreams; /* should be >= 16384 and an even number of MSS for $QIOing */ h2ptr->ServerMaxFrameSize = h2ptr->NetIoPtr->TcpMaxQio * 2; while (h2ptr->ServerMaxFrameSize * 2 > h2ptr->NetIoPtr->TcpSndBuf || h2ptr->ServerMaxFrameSize * 2 > h2ptr->NetIoPtr->TcpRcvBuf) h2ptr->ServerMaxFrameSize -= h2ptr->NetIoPtr->TcpMaxSeg; if (h2ptr->ServerMaxFrameSize < HTTP2_INITIAL_MAX_FRAME_SIZE) h2ptr->ServerMaxFrameSize = HTTP2_INITIAL_MAX_FRAME_SIZE; h2ptr->ServerMaxHeaderListSize = Http2MaxHeaderListSize; h2ptr->ServerMaxHeaderTableSize = Http2MaxHeaderTableSize; h2ptr->ServerPushPromise = 0; /* intialise client protocol parameters */ h2ptr->ClientInitialWindowSize = HTTP2_INITIAL_WINDOW_SIZE; h2ptr->ClientMaxConcStreams = HTTP2_INITIAL_MAX_CONC_STREAMS; h2ptr->ClientMaxFrameSize = HTTP2_INITIAL_MAX_FRAME_SIZE; h2ptr->ClientMaxHeaderListSize = HTTP2_INITIAL_HEAD_LIST_SIZE; h2ptr->ClientMaxHeaderTableSize = HTTP2_INITIAL_HEAD_TAB_SIZE; h2ptr->ClientPushPromise = 0; /* HPACK initialisation */ h2ptr->HpackClientTable.max = Http2MaxHeaderTableSize; h2ptr->HpackServerTable.max = Http2MaxHeaderTableSize; h2ptr->HpackClientTable.h2ptr = h2ptr->HpackServerTable.h2ptr = h2ptr; /* initial flow control */ h2ptr->ReadWindowSize = h2ptr->ServerInitialWindowSize; h2ptr->WriteWindowSize = h2ptr->ClientInitialWindowSize; /* send server settings */ Http2Settings (h2ptr, 0, 0, NULL, 0); /* set the initial connection flow-control window size */ h2ptr->ReadWindowSize = h2ptr->ServerInitialWindowSize * h2ptr->ServerMaxConcStreams; Http2WindowUpdate (h2ptr, NULL, 0, h2ptr->ReadWindowSize, NULL, 0); /* buffer for multiplexed reads */ h2ptr->ReadBufferSize = h2ptr->ServerMaxFrameSize + 128; h2ptr->ReadBufferPtr = VmGet2Heap (h2ptr, h2ptr->ReadBufferSize); return (h2ptr); } /*****************************************************************************/ /* Close the HTTP/2 protocol connection with the client. If there are still requests associated with the connection wait for them to end (based on delivered CANCELed I/O). If there's still HTTP/2 I/O in progress wait for it to rundown. Then check for any connection management writes still queued. Finally a check for straggling connection I/O in progress. After that the connection is removed from the list and the associated memory released in an indepedent AST. This function should only be called the once by HTTP/2 processing code when the channel number is non-zero. After that the Http2Supervisor() calls it every second until all the above conditions are met. Once removed from the connection list it's gone! */ void Http2ConnectClose (HTTP2_STRUCT *h2ptr) { BOOL OneShot; int idx; HTTP2_STREAM_STRUCT *s2ptr, *s22ptr; /*********/ /* begin */ /*********/ if (WATCHMOD (h2ptr, WATCH_MOD_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_MOD_HTTP2, "Http2ConnectClose() !&F chan:!UL list:!UL read:!&B (!UL) write:!&B (!UL)", Http2ConnectClose, h2ptr->NetIoPtr->Channel, LIST_GET_COUNT (&h2ptr->StreamList), NETIO_READ_IN_PROGRESS(h2ptr->NetIoPtr), h2ptr->ReadInUseCount, NETIO_WRITE_IN_PROGRESS(h2ptr->NetIoPtr), h2ptr->WriteInUseCount); h2ptr->ConnectClosing = true; h2ptr->NetIoPtr->VmsStatus = SS$_VCCLOSED; if (LIST_GET_COUNT (&h2ptr->StreamList)) { /* still active stream(s) */ for (s2ptr = LIST_GET_HEAD(&h2ptr->StreamList); s2ptr != NULL; s2ptr = s22ptr) { /* otherwise we'll be slipping on the bathmat */ s22ptr = LIST_GET_NEXT(s2ptr); if (!s2ptr->StreamClosed) Http2StreamDrop (s2ptr); } if (LIST_GET_COUNT (&h2ptr->StreamList)) return; } /* any read still in progress */ if (NETIO_READ_IN_PROGRESS(h2ptr->NetIoPtr)) { sys$cancel (h2ptr->NetIoPtr->Channel); return; } /* still outstanding client read */ if (h2ptr->ReadInUseCount) return; /* if still queued write(s) waiting */ for (idx = HTTP2_WRITE_QUEUE_MAX; idx <= HTTP2_WRITE_QUEUE_LOW; idx++) if (LIST_NOT_EMPTY (&h2ptr->QueuedWriteList[idx])) { Http2NetQueueWrite (h2ptr, NULL); return; } /* any write still in progress */ if (NETIO_WRITE_IN_PROGRESS(h2ptr->NetIoPtr)) return; if (h2ptr->WriteInUseCount > 1) { char buf [64]; FaoToBuffer (buf, sizeof(buf), NULL, "WriteInUseCount:!SL", h2ptr->WriteInUseCount); ErrorNoticed (NULL, 0, buf, FI_LI); } if (h2ptr->WriteInUseCount > 0) return; /***********/ /* finally */ /***********/ if (WATCHPNT(h2ptr)) { if (WATCH_CATEGORY(WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "FRAMES tx:!UL rx:!UL REQUESTS tx:!UL rx:!UL total:!UL peak:!UL", h2ptr->FrameCountTx, h2ptr->FrameCountRx, h2ptr->FrameRequestCountTx, h2ptr->FrameRequestCountRx, h2ptr->RequestCount, h2ptr->RequestPeak); if (WATCH_CATEGORY(WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "HTTP/2 closed"); else if (WATCH_CATEGORY(WATCH_CONNECT)) WatchThis (WATCHITM(h2ptr), WATCH_CONNECT, "HTTP/2 closed"); } NetIoEnd (h2ptr->NetIoPtr); InstanceMutexLock (INSTANCE_MUTEX_HTTPD); AccountingPtr->Http2FlowFrameCount64 += h2ptr->FlowFrameTally; AccountingPtr->Http2FlowStallCount64 += h2ptr->FlowStallTally; NetUpdateConnected (h2ptr, -1); InstanceMutexUnLock (INSTANCE_MUTEX_HTTPD); /* remove from the HTTP/2 connection list */ ListRemove (&Http2List, h2ptr); VmFree2Heap (h2ptr, FI_LI); /* if this HTTP/2 connection was being (one-shot) WATCHed */ OneShot = h2ptr->WatchItem & WATCH_ITEM_HTTP2_FLAG; VmHttp2Free (h2ptr, FI_LI); if (OneShot && Watch.RequestPtr) { Watch.RequestPtr->RequestState = REQUEST_STATE_ENDING; SysDclAst (RequestEnd, Watch.RequestPtr); } } /*****************************************************************************/ /* Create a new idle HTTP/2 stream. */ HTTP2_STREAM_STRUCT* Http2StreamGet ( HTTP2_STRUCT *h2ptr, uint ident ) { HTTP2_STREAM_STRUCT *s2ptr; /*********/ /* begin */ /*********/ if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "STREAM !UL (!UL) get !UL", h2ptr->ConnectNumber, LIST_GET_COUNT(&h2ptr->StreamList), h2ptr->StreamCount + 1); s2ptr = VmGet2Heap (h2ptr, sizeof(HTTP2_STREAM_STRUCT)); ListAddHead (&h2ptr->StreamList, s2ptr, LIST_ENTRY_TYPE_REQUEST); s2ptr->Http2Ptr = h2ptr; s2ptr->Ident = ident; h2ptr->LastStreamIdent = ident; s2ptr->StreamNumber = ++h2ptr->StreamCount; /* set the initial stream flow-control window size */ s2ptr->ReadWindowSize = h2ptr->ServerInitialWindowSize; s2ptr->WriteWindowSize = h2ptr->WriteWindowSize; /* inform the client of this */ Http2WindowUpdate (h2ptr, s2ptr, s2ptr->Ident, s2ptr->ReadWindowSize, NULL, 0); return (s2ptr); } /*****************************************************************************/ /* Remove an HTTP/2 stream. Wait for any in-progress request to complete. The data s2ptr->FlowStallTickSecond and h2ptr->FlowStallTickSecond will occasionally contain a value 1 (instead of a legitimate HttpTickSecond) and is used to trigger a dropped stream. */ HTTP2_STREAM_STRUCT* Http2StreamDrop (HTTP2_STREAM_STRUCT *s2ptr) { HTTP2_STRUCT *h2ptr; REQUEST_STRUCT *rqptr; /*********/ /* begin */ /*********/ h2ptr = s2ptr->Http2Ptr; if (WATCHMOD (h2ptr, WATCH_MOD_HTTP2)) { if (rqptr = s2ptr->RequestPtr) WatchThis (WATCHITM(h2ptr), WATCH_MOD_HTTP2, "Http2StreamDrop() !UL !UL/!UL (!UL) rqptr:!UL (!UL !AZ)", s2ptr->Ident, h2ptr->ConnectNumber, s2ptr->StreamNumber, LIST_GET_COUNT(&h2ptr->StreamList), rqptr->ConnectNumber, rqptr->RequestState, RequestState(rqptr->RequestState)); else WatchThis (WATCHITM(h2ptr), WATCH_MOD_HTTP2, "Http2StreamDrop() !UL !UL/!UL (!UL)", s2ptr->Ident, h2ptr->ConnectNumber, s2ptr->StreamNumber, LIST_GET_COUNT(&h2ptr->StreamList)); } s2ptr->StreamClosed = true; if (rqptr = s2ptr->RequestPtr) { if (!s2ptr->FlowStallTickSecond) s2ptr->FlowStallTickSecond = HTTP2_STREAM_DROPPED; else if (s2ptr->FlowStallTickSecond && s2ptr->FlowStallTickSecond < HttpdTickSecond) { if (s2ptr->FlowStallTickSecond > HTTP2_STREAM_DROPPED) { Http2FlowCheck (h2ptr, s2ptr, NULL); Http2RequestResetStream (rqptr); s2ptr->FlowStallTickSecond = HTTP2_STREAM_DROPPED; rqptr->rqResponse.HttpStatus = 422; } } else if (h2ptr->FlowStallTickSecond && h2ptr->FlowStallTickSecond < HttpdTickSecond) { if (h2ptr->FlowStallTickSecond > HTTP2_STREAM_DROPPED) { Http2FlowCheck (h2ptr, s2ptr, NULL); if (!s2ptr->FlowStallTickSecond) Http2RequestResetStream (rqptr); } h2ptr->FlowStallTickSecond = HTTP2_STREAM_DROPPED; rqptr->rqResponse.HttpStatus = 422; } if (rqptr->RequestState < REQUEST_STATE_PROCESSING) Http2RequestEnd5 (rqptr); else if (!Http2NetCancelWrite (rqptr)) if (rqptr->RequestState >= REQUEST_STATE_ABORT) Http2RequestEnd4 (rqptr); /* for as long as a request is attached to the stream */ return (s2ptr); } ListRemove (&h2ptr->StreamList, s2ptr); VmFreeFrom2Heap (h2ptr, s2ptr, FI_LI); return (NULL); } /*****************************************************************************/ /* Mark a stream as ended and ready to be dropped. */ void Http2StreamClose (HTTP2_STREAM_STRUCT *s2ptr) { HTTP2_STRUCT *h2ptr; REQUEST_STRUCT *rqptr; /*********/ /* begin */ /*********/ h2ptr = s2ptr->Http2Ptr; if (WATCHING (h2ptr, WATCH_HTTP2)) { if (rqptr = s2ptr->RequestPtr) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "STREAM !UL close !UL/!UL rqptr:!UL (!UL !AZ)", s2ptr->Ident, h2ptr->ConnectNumber, s2ptr->StreamNumber, rqptr->ConnectNumber, rqptr->RequestState, RequestState(rqptr->RequestState)); else WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "STREAM !UL close !UL/!UL (!UL)", s2ptr->Ident, h2ptr->ConnectNumber, s2ptr->StreamNumber, LIST_GET_COUNT(&h2ptr->StreamList)); } s2ptr->StreamClosed = true; s2ptr->DropTickSecond = HttpdTickSecond + HTTP2_DROP_SECONDS; if (rqptr = s2ptr->RequestPtr) { /* special case - equivalent to HTTP/1.n WatchBreakDetect() */ if (rqptr == Watch.RequestPtr) WatchEnd (); } } /*****************************************************************************/ /* After completion streams are kept around for small time to allow straggling frames to find the stream and be dealt with accordingly (usually error). Http2Supervisor() calls this function each second. */ void Http2StreamPurge (HTTP2_STRUCT *h2ptr) { HTTP2_STREAM_STRUCT *s2ptr, *s22ptr; /*********/ /* begin */ /*********/ if (WATCHMOD (h2ptr, WATCH_MOD_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_MOD_HTTP2, "Http2StreamPurge()"); for (s2ptr = LIST_GET_HEAD(&h2ptr->StreamList); s2ptr != NULL; s2ptr = s22ptr) { /* otherwise we'll be slipping on the bathmat */ s22ptr = LIST_GET_NEXT(s2ptr); if (s2ptr->DropTickSecond && s2ptr->DropTickSecond < HttpdTickSecond) Http2StreamDrop (s2ptr); } } /*****************************************************************************/ /* With flow control stalls a real possibility periodically purge the exhausted. Http2Supervisor() calls this function each second. */ void Http2StreamFlow (HTTP2_STRUCT *h2ptr) { HTTP2_STREAM_STRUCT *s2ptr, *s22ptr; /*********/ /* begin */ /*********/ if (WATCHMOD (h2ptr, WATCH_MOD_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_MOD_HTTP2, "Http2StreamFlow()"); for (s2ptr = LIST_GET_HEAD(&h2ptr->StreamList); s2ptr != NULL; s2ptr = s22ptr) { /* otherwise we'll be slipping on the bathmat */ s22ptr = LIST_GET_NEXT(s2ptr); if (!s2ptr->FlowStallTickSecond) continue; if (s2ptr->FlowStallTickSecond < HttpdTickSecond) if (Http2FlowCheck (h2ptr, s2ptr, NULL) < 0) Http2StreamDrop (s2ptr); } } /*****************************************************************************/ /* An HTTP/2 error has been encountered. Signal this to the client as a non-fatal stream error or a fatal connection error. Return zero to continue, negative to abort the connection. */ int Http2DoError ( HTTP2_STRUCT *h2ptr, uint ident, uint error, char *module, int line ) { int number, retval; char *cptr; HTTP2_STREAM_STRUCT *s2ptr; /*********/ /* begin */ /*********/ if (WATCHMOD (h2ptr, WATCH_MOD_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_MOD_HTTP2, "Http2DoError() !SL", error); if (WATCHING (h2ptr, WATCH_HTTP2)) Http2Error (h2ptr, error, module, line); if ((number = error) < 0) number = -number; switch (number) { /* stream oriented non-fatal errors */ case HTTP2_ERROR_NONE : case HTTP2_ERROR_CANCEL : case HTTP2_ERROR_CALM : case HTTP2_ERROR_HTTP11 : case HTTP2_ERROR_CLOSED : for (s2ptr = LIST_GET_HEAD(&h2ptr->StreamList); s2ptr != NULL; s2ptr = LIST_GET_NEXT(s2ptr)); if (s2ptr) { retval = Http2StreamReset (h2ptr, s2ptr, number, NULL, 0); if (retval >= 0) return (0); } Http2GoAway (h2ptr, 0, -(int)number, NULL, 0); return (-(int)number); /* connection breaking errors */ case HTTP2_ERROR_PROTOCOL : case HTTP2_ERROR_REFUSED : case HTTP2_ERROR_INTERNAL : case HTTP2_ERROR_FLOW : case HTTP2_ERROR_TIMEOUT : case HTTP2_ERROR_SIZE : case HTTP2_ERROR_COMPRESS : case HTTP2_ERROR_CONNECT : case HTTP2_ERROR_SECURITY : Http2GoAway (h2ptr, 0, number, NULL, 0); return (-(int)number); default : Http2GoAway (h2ptr, 0, HTTP2_ERROR_INTERNAL, NULL, 0); return (-HTTP2_ERROR_INTERNAL); } } /*****************************************************************************/ /* Report the error, including the code origin, as well as returning it. */ int Http2Error ( HTTP2_STRUCT *h2ptr, int error, char *module, int line ) { int number; char *sptr; /*********/ /* begin */ /*********/ if (!(h2ptr && h2ptr->WatchItem)) return (error); if ((number = error) < 0) number = -number; if (number < HTTP2_ERROR_COUNT) sptr = Http2ErrorArray[number]; else sptr = "?"; WatchThis (WATCHALL, WATCH_HTTP2, "ERROR !UL !AZ !AZ:!UL", number, sptr, module, line); return (error); } /*****************************************************************************/ /* Send and receive GOAWAY frames. */ int Http2GoAway ( HTTP2_STRUCT *h2ptr, uint ident, uint error, uchar *bptr, uint blen ) { uint size, stream; uchar *cptr, *czptr, *sptr, *zptr; uchar buf [256]; HTTP2_WRITE_STRUCT *w2ptr; /*********/ /* begin */ /*********/ if (WATCHMOD (h2ptr, WATCH_MOD_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_MOD_HTTP2, "Http2GoAway() bptr:!8XL blen:!UL closing:!&B", bptr, blen, h2ptr->ConnectClosing); if (h2ptr->ConnectClosing) return (0); if (ident) return (Http2Error(h2ptr,-HTTP2_ERROR_PROTOCOL,FI_LI)); if (bptr) { /**********************/ /* goaway from client */ /**********************/ if (blen < 8) return (Http2Error(h2ptr,-HTTP2_ERROR_SIZE,FI_LI)); HTTP2_GET_32 (bptr, stream); stream &= 0x7fffffff; HTTP2_GET_32 (bptr, error); zptr = (sptr = buf) + sizeof(buf)-1; if (blen > 8) { czptr = (cptr = bptr) + blen - 8; while (cptr < czptr && sptr < zptr) *sptr++ = *cptr++; } *sptr = '\0'; if (stream) return (Http2Error(h2ptr,-HTTP2_ERROR_PROTOCOL,FI_LI)); } else { /********************/ /* goaway to client */ /********************/ h2ptr->GoAwayIdent = h2ptr->LastStreamIdent; if (!bptr) bptr = ""; for (cptr = bptr; *cptr; cptr++); size = cptr - bptr; if (size <= (HTTP2_WRITE_PAYLOAD - 8)) size = 0; w2ptr = Http2GetWriteStruct (h2ptr, size, FI_LI); sptr = bptr; bptr = w2ptr->payload; HTTP2_PUT_32 (bptr, h2ptr->GoAwayIdent); if ((int)error < 0) error = -error; HTTP2_PUT_32 (bptr, error); while (*bptr) *bptr++ = *sptr++; *bptr = '\0'; HTTP2_PLACE_24 (w2ptr->length, bptr - w2ptr->payload); HTTP2_PLACE_8 (w2ptr->type, HTTP2_FRAME_GOAWAY); HTTP2_PLACE_8 (w2ptr->flags, 0); HTTP2_PLACE_32 (w2ptr->ident, 0); w2ptr->Http2Ptr = h2ptr; w2ptr->AstParam = h2ptr; /* send the goaway frame and once sent close the connection */ w2ptr->AstFunction = Http2ConnectClose; w2ptr->HeaderPtr = w2ptr->header; w2ptr->DataPtr = w2ptr->payload; w2ptr->DataLength = bptr - w2ptr->payload; if (WATCHING (h2ptr, WATCH_HTTP2)) Http2WatchGoAway (h2ptr, w2ptr->DataPtr, w2ptr->DataLength); Http2NetQueueWrite (h2ptr, w2ptr); } h2ptr->GoAwayLastStreamIdent = stream; /* if not gone after this period then close the connection anyway */ h2ptr->GoAwayTickSecond = HttpdTickSecond + HTTP2_TIMEOUT_GOAWAY_SECONDS; return (0); } /*****************************************************************************/ /* Send and receive PING frames. */ int Http2Ping ( HTTP2_STRUCT *h2ptr, uint ident, uint flags, uchar *bptr, uint blen ) { uchar *sptr; int64 ResultTime64; HTTP2_WRITE_STRUCT *w2ptr; /*********/ /* begin */ /*********/ if (WATCHMOD (h2ptr, WATCH_MOD_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_MOD_HTTP2, "Http2Ping() blen:!UL closing:!&B", blen, h2ptr->ConnectClosing); if (h2ptr->ConnectClosing) return (0); if (bptr) { /********************/ /* ping from client */ /********************/ if (ident) return (Http2Error(h2ptr,-HTTP2_ERROR_PROTOCOL,FI_LI)); if (blen != 8) return (Http2Error(h2ptr,-HTTP2_ERROR_SIZE,FI_LI)); if (flags & HTTP2_FLAG_ACK) { /*******************/ /* ack from client */ /*******************/ /* client acknowledging server ping */ if (*(INT64PTR)bptr == h2ptr->PingTime64) { /* the number of 100 nano-seconds (up to 100 seconds) */ sys$gettim (&ResultTime64); ResultTime64 -= h2ptr->PingTime64; h2ptr->PingMicroSeconds = ResultTime64 / 10; } else h2ptr->PingMicroSeconds = -1; if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, (int)h2ptr->PingMicroSeconds == -1 ? "PING client ERROR" : "PING !UL.!3ZLmS", h2ptr->PingMicroSeconds / 1000, h2ptr->PingMicroSeconds % 1000); h2ptr->PingBackTickSecond = 0; h2ptr->PingTime64 = 0; return (0); } /*****************/ /* ack to client */ /*****************/ if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "PING server ACKNOWLEDGE"); /* mitigate CVE-2019-9512 */ if (h2ptr->PingLimitCount++ > HTTP2_PING_LIMIT_COUNT) { if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "LIMIT client ping to !UL/S (DoS?)", HTTP2_PING_LIMIT_COUNT); return (Http2Error(h2ptr,-HTTP2_ERROR_CALM,FI_LI)); } w2ptr = Http2GetWriteStruct (h2ptr, 0, FI_LI); /* replay the client payload */ sptr = w2ptr->payload; while (blen--) *sptr++ = *bptr++; HTTP2_PLACE_24 (w2ptr->length, 8); HTTP2_PLACE_8 (w2ptr->type, HTTP2_FRAME_PING); HTTP2_PLACE_8 (w2ptr->flags, HTTP2_FLAG_ACK); HTTP2_PLACE_32 (w2ptr->ident, 0); w2ptr->Http2Ptr = h2ptr; w2ptr->HeaderPtr = w2ptr->header; w2ptr->DataPtr = w2ptr->payload; w2ptr->DataLength = 8; } else { /******************/ /* ping to client */ /******************/ /* only one outstanding with the specified timeout */ if (h2ptr->PingBackTickSecond) if (HttpdTickSecond < h2ptr->PingBackTickSecond) return (0); w2ptr = Http2GetWriteStruct (h2ptr, 0, FI_LI); /* generate and note the ping time */ sys$gettim (&h2ptr->PingTime64); *(INT64PTR)w2ptr->payload = h2ptr->PingTime64; h2ptr->PingBackTickSecond = HttpdTickSecond + HTTP2_PING_RESPONSE_MAX; HTTP2_PLACE_24 (w2ptr->length, 8); HTTP2_PLACE_8 (w2ptr->type, HTTP2_FRAME_PING); HTTP2_PLACE_8 (w2ptr->flags, 0); HTTP2_PLACE_32 (w2ptr->ident, 0); w2ptr->Http2Ptr = h2ptr; w2ptr->HeaderPtr = w2ptr->header; w2ptr->DataPtr = w2ptr->payload; w2ptr->DataLength = 8; if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "PING to client"); } Http2NetQueueWrite (h2ptr, w2ptr); return (0); } /*****************************************************************************/ /* HTTP/2 priority/dependency is not currently implemented. Weight is weight plus one, 1..256. */ int Http2Priority ( HTTP2_STRUCT *h2ptr, uint ident, uint flags, uchar *bptr, uint blen ) { uint depend, value, weight; /*********/ /* begin */ /*********/ if (WATCHMOD (h2ptr, WATCH_MOD_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_MOD_HTTP2, "Http2Priority() blen:!UL closing:!&B", blen, h2ptr->ConnectClosing); if (h2ptr->ConnectClosing) return (0); if (h2ptr->HeaderThisIdent) return (Http2Error(h2ptr,-HTTP2_ERROR_PROTOCOL,FI_LI)); if (blen != 5) return (Http2Error(h2ptr,-HTTP2_ERROR_SIZE,FI_LI)); HTTP2_GET_32 (bptr, depend); if (depend == ident) return (Http2Error(h2ptr,-HTTP2_ERROR_PROTOCOL,FI_LI)); HTTP2_GET_8 (bptr, weight); return (0); } /*****************************************************************************/ /* Mark the stream as closed. When the server closes a stream (as when a request times out) send a RST_STREAM frame to the client. */ int Http2StreamReset ( HTTP2_STRUCT *h2ptr, HTTP2_STREAM_STRUCT *s2ptr, uint error, uchar *bptr, uint blen ) { HTTP2_WRITE_STRUCT *w2ptr; /*********/ /* begin */ /*********/ if (WATCHMOD (h2ptr, WATCH_MOD_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_MOD_HTTP2, "Http2StreamReset() blen:!UL ident:!UL error:!UL open:!&B closing:!&B", blen, s2ptr ? s2ptr->Ident : 0, error, s2ptr ? s2ptr->StreamOpen : false, h2ptr->ConnectClosing); /* if nonexistant stream */ if (!s2ptr) return (Http2Error(h2ptr,-HTTP2_ERROR_PROTOCOL,FI_LI)); /* RFC9113 6.4 */ if (!s2ptr->StreamOpen) return (Http2Error(h2ptr,-HTTP2_ERROR_PROTOCOL,FI_LI)); if (bptr) { /*******************/ /* reset by client */ /*******************/ /* RFC9113 6.4 */ if (blen != 4) return (Http2Error(h2ptr,-HTTP2_ERROR_SIZE,FI_LI)); /* keep count for CVE-2023-44487 */ h2ptr->RstStreamCount++; /* mitigate CVE-2019-9514 */ if (h2ptr->StreamResetLimitCount++ > HTTP2_STREAM_RESET_LIMIT_COUNT) { s2ptr->StreamClosed = true; if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "LIMIT server stream resets to !UL/S (DoS?)", HTTP2_STREAM_RESET_LIMIT_COUNT); return (Http2Error(h2ptr,-HTTP2_ERROR_CALM,FI_LI)); } Http2StreamClose (s2ptr); } else { /*******************/ /* reset by server */ /*******************/ w2ptr = Http2GetWriteStruct (h2ptr, 0, FI_LI); HTTP2_PLACE_24 (w2ptr->length, 4); HTTP2_PLACE_8 (w2ptr->type, HTTP2_FRAME_RST_STREAM); HTTP2_PLACE_8 (w2ptr->flags, 0); HTTP2_PLACE_32 (w2ptr->ident, s2ptr->Ident); w2ptr->Http2Ptr = h2ptr; w2ptr->HeaderPtr = w2ptr->header; w2ptr->DataPtr = w2ptr->payload; w2ptr->DataLength = 4; HTTP2_PLACE_32 (w2ptr->payload, error); /* AST to (try and) drop the stream */ w2ptr->AstFunction = Http2StreamClose; w2ptr->AstParam = s2ptr; Http2NetQueueWrite (h2ptr, w2ptr); } return (0); } /*****************************************************************************/ /* Send and receive HTTP/2 settings. */ int Http2Settings ( HTTP2_STRUCT *h2ptr, uint ident, uint flags, uchar *bptr, uint blen ) { uint count, setting, value; uchar *aptr; HTTP2_WRITE_STRUCT *w2ptr; /*********/ /* begin */ /*********/ if (WATCHMOD (h2ptr, WATCH_MOD_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_MOD_HTTP2, "Http2Settings() ident:!UL flags:0x!2XL 0x!8XL len:!UL (!UL) closing:!&B", ident, flags, bptr, blen, blen / 6, h2ptr->ConnectClosing); if (h2ptr->ConnectClosing) return (0); if (ident) return (Http2Error(h2ptr,-HTTP2_ERROR_PROTOCOL,FI_LI)); if (bptr) { /***************/ /* from client */ /***************/ /* all setting are in multiples of 6 octets */ if (blen % 6) return (Http2Error(h2ptr,-HTTP2_ERROR_SIZE,FI_LI)); count = blen / 6; if (flags & HTTP2_FLAG_ACK) { /**************/ /* client ack */ /**************/ /* payload must be empty */ if (count) return (Http2Error(h2ptr,-HTTP2_ERROR_PROTOCOL,FI_LI)); return (0); } /*********************/ /* client setting(s) */ /*********************/ /* mitigate CVE-2019-9515 */ if (h2ptr->SettingsLimitCount++ > HTTP2_SETTINGS_LIMIT_COUNT) { if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "LIMIT client settings to !UL/S (DoS?)", HTTP2_SETTINGS_LIMIT_COUNT); return (Http2Error(h2ptr,-HTTP2_ERROR_CALM,FI_LI)); } while (count--) { HTTP2_GET_16 (bptr, setting); HTTP2_GET_32 (bptr, value); /* process client requested settings */ switch (setting) { case HTTP2_SETTING_MAX_HEAD_TABLE_SIZE : h2ptr->ClientMaxHeaderTableSize = value; h2ptr->HpackClientTable.max = value; break; case HTTP2_SETTING_ENABLE_PUSH : if (!(value == 0 || value == 1)) return (Http2Error(h2ptr,-HTTP2_ERROR_PROTOCOL,FI_LI)); h2ptr->ClientPushPromise = value; break; case HTTP2_SETTING_MAX_CONC_STREAMS : h2ptr->ClientMaxConcStreams = value; break; case HTTP2_SETTING_INIT_WIN_SIZE : if (value >= 2147483648) return (Http2Error(h2ptr,-HTTP2_ERROR_FLOW,FI_LI)); Http2FlowControl (h2ptr, value); break; case HTTP2_SETTING_MAX_FRAME_SIZE : if (value < HTTP2_MIN_FRAME_SIZE) return (Http2Error(h2ptr,-HTTP2_ERROR_PROTOCOL,FI_LI)); if (value > HTTP2_MAX_FRAME_SIZE) return (Http2Error(h2ptr,-HTTP2_ERROR_PROTOCOL,FI_LI)); h2ptr->ClientMaxFrameSize = value; break; case HTTP2_SETTING_MAX_HEAD_LIST_SIZE : h2ptr->ClientMaxHeaderListSize = value; break; default : /* ignore unknown settings RFC 7540 6.5.2 */ } } /*****************/ /* ack to client */ /*****************/ if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "SETTINGS server ACKNOWLEDGE"); w2ptr = Http2GetWriteStruct (h2ptr, 0, FI_LI); HTTP2_PLACE_24 (w2ptr->length, 0); HTTP2_PLACE_8 (w2ptr->type, HTTP2_FRAME_SETTINGS); HTTP2_PLACE_8 (w2ptr->flags, HTTP2_FLAG_ACK); HTTP2_PLACE_32 (w2ptr->ident, 0); w2ptr->Http2Ptr = h2ptr; w2ptr->HeaderPtr = w2ptr->header; w2ptr->DataPtr = w2ptr->payload; w2ptr->DataLength = 0; } else { /*************/ /* to client */ /*************/ w2ptr = Http2GetWriteStruct (h2ptr, 0, FI_LI); bptr = w2ptr->payload; HTTP2_PUT_16 (bptr, HTTP2_SETTING_MAX_HEAD_TABLE_SIZE); HTTP2_PUT_32 (bptr, h2ptr->ServerMaxHeaderTableSize); HTTP2_PUT_16 (bptr, HTTP2_SETTING_MAX_CONC_STREAMS); HTTP2_PUT_32 (bptr, h2ptr->ServerMaxConcStreams); HTTP2_PUT_16 (bptr, HTTP2_SETTING_INIT_WIN_SIZE); HTTP2_PUT_32 (bptr, h2ptr->ServerInitialWindowSize); HTTP2_PUT_16 (bptr, HTTP2_SETTING_MAX_FRAME_SIZE); HTTP2_PUT_32 (bptr, h2ptr->ServerMaxFrameSize); HTTP2_PUT_16 (bptr, HTTP2_SETTING_MAX_HEAD_LIST_SIZE); HTTP2_PUT_32 (bptr, h2ptr->ServerMaxHeaderListSize); HTTP2_PLACE_24 (w2ptr->length, bptr - w2ptr->payload); HTTP2_PLACE_8 (w2ptr->type, HTTP2_FRAME_SETTINGS); HTTP2_PLACE_8 (w2ptr->flags, 0); HTTP2_PLACE_32 (w2ptr->ident, 0); w2ptr->Http2Ptr = h2ptr; w2ptr->HeaderPtr = w2ptr->header; w2ptr->DataPtr = w2ptr->payload; w2ptr->DataLength = bptr - w2ptr->payload; if (WATCHING (h2ptr, WATCH_HTTP2)) Http2WatchSettings (h2ptr, "server", w2ptr->DataPtr, w2ptr->DataLength); } Http2NetQueueWrite (h2ptr, w2ptr); return (0); } /*****************************************************************************/ /* Client setting change to initial window size initiates connection-wide flow control window adjustment. The resulting window size can be negative (RFC 7540 6.9.2). Blocked flow control may now resume, or vice-versa. */ void Http2FlowControl ( HTTP2_STRUCT *h2ptr, uint value ) { int adjust; HTTP2_STREAM_STRUCT *s2ptr; /*********/ /* begin */ /*********/ if (WATCHMOD (h2ptr, WATCH_MOD_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_MOD_HTTP2, "Http2FlowControl() !UL", value); if (value < h2ptr->ClientInitialWindowSize) adjust = value - h2ptr->ClientInitialWindowSize; else if (value > h2ptr->ClientInitialWindowSize) adjust = h2ptr->ClientInitialWindowSize - value; else return; h2ptr->ClientInitialWindowSize = value; h2ptr->WriteWindowSize += adjust; if (!adjust) return; for (s2ptr = LIST_GET_HEAD(&h2ptr->StreamList); s2ptr != NULL; s2ptr = LIST_GET_NEXT(s2ptr)) s2ptr->WriteWindowSize += adjust; if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "FLOW adjust !SL to !SL", adjust, h2ptr->WriteWindowSize); /* resume writing if flow control was blocking and now permits */ if (!NETIO_WRITE_IN_PROGRESS(h2ptr->NetIoPtr)) Http2NetQueueWrite (h2ptr, NULL); } /*****************************************************************************/ /* Flow control feedback from the client. */ int Http2WindowUpdate ( HTTP2_STRUCT *h2ptr, HTTP2_STREAM_STRUCT *s2ptr, uint ident, uint update, uchar *bptr, uint blen ) { uint size; HTTP2_WRITE_STRUCT *w2ptr; /*********/ /* begin */ /*********/ if (WATCHMOD (h2ptr, WATCH_MOD_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_MOD_HTTP2, "Http2WindowUpdate() blen:!UL ident:!UL", blen, ident); if (bptr) { /***************/ /* from client */ /***************/ if (s2ptr) if (!s2ptr->StreamOpen) return (Http2Error(h2ptr,-HTTP2_ERROR_PROTOCOL,FI_LI)); if (blen != 4) return (Http2Error(h2ptr,-HTTP2_ERROR_SIZE,FI_LI)); HTTP2_GET_32 (bptr, update); if (WATCHING (h2ptr, WATCH_HTTP2) && !WATCHING1S(h2ptr) && !WATCHING2(h2ptr)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "WINDOW client ident:!UL update:!UL", ident, update); if (update == 0 || update & 0x80000000) return (Http2Error(h2ptr,-HTTP2_ERROR_PROTOCOL,FI_LI)); if ((s2ptr && s2ptr->FlowStallTickSecond) || (h2ptr && h2ptr->FlowStallTickSecond)) Http2FlowCheck (h2ptr, s2ptr, NULL); if (ident) { /* stream */ size = (s2ptr->WriteWindowSize += update); } else { /* connection */ size = (h2ptr->WriteWindowSize += update); } if (WATCHING (h2ptr, WATCH_HTTP2) && !WATCHING1S(h2ptr) && !WATCHING2(h2ptr)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "WINDOW size !SL", s2ptr ? s2ptr->WriteWindowSize : h2ptr->WriteWindowSize); /* resume writing if flow control was blocking and now permits */ if (!NETIO_WRITE_IN_PROGRESS(h2ptr->NetIoPtr)) Http2NetQueueWrite (h2ptr, NULL); } else { /*************/ /* to client */ /*************/ if (WATCHING (h2ptr, WATCH_HTTP2) && !WATCHING1S(h2ptr) && !WATCHING2(h2ptr)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "WINDOW server ident:!UL update:!UL", ident, update); if (update == 0 || update & 0x80000000) { /* note and ignore */ ErrorNoticed (NULL, SS$_BUGCHECK, ErrorSanityCheck, FI_LI); return (0); } w2ptr = Http2GetWriteStruct (h2ptr, 0, FI_LI); HTTP2_PLACE_24 (w2ptr->length, 4); HTTP2_PLACE_8 (w2ptr->type, HTTP2_FRAME_WINDOW_UPDATE); HTTP2_PLACE_8 (w2ptr->flags, 0); HTTP2_PLACE_32 (w2ptr->ident, ident); HTTP2_PLACE_32 (w2ptr->payload, update); w2ptr->Http2Ptr = h2ptr; w2ptr->HeaderPtr = w2ptr->header; w2ptr->DataPtr = w2ptr->payload; w2ptr->DataLength = 4; Http2NetQueueWrite (h2ptr, w2ptr); } return (0); } /*****************************************************************************/ /* Allocate a write struct while performing some sanity checking. */ HTTP2_WRITE_STRUCT* Http2GetWriteStruct ( HTTP2_STRUCT *h2ptr, int size, char *module, int line ) { HTTP2_WRITE_STRUCT *w2ptr; /*********/ /* begin */ /*********/ if (++h2ptr->WriteInUseCount > 256) /* arbitrary */ { char *cptr = FaoToMemory (NULL, "WriteInUseCount:!SL", h2ptr->WriteInUseCount); ErrorNoticed (NULL, SS$_BUGCHECK, cptr, module, line); VmFree (cptr, FI_LI); } w2ptr = VmGet2Heap (h2ptr, sizeof(HTTP2_WRITE_STRUCT) + size); return (w2ptr); } /*****************************************************************************/ /* Free a write struct while performing some sanity checking. */ void Http2FreeWriteStruct ( HTTP2_STRUCT *h2ptr, HTTP2_WRITE_STRUCT *w2ptr, char *module, int line ) { /*********/ /* begin */ /*********/ if (--h2ptr->WriteInUseCount < 0) { HttpdStackTrace ("Http2FreeWriteStruct()", FI_LI); char *cptr = FaoToMemory (NULL, "WriteInUseCount:!SL", h2ptr->WriteInUseCount); ErrorNoticed (NULL, SS$_BUGCHECK, cptr, module, line); VmFree (cptr, FI_LI); } VmFreeFrom2Heap (h2ptr, w2ptr, module, line); } /*****************************************************************************/ /* Return a pointer to a string respresenting the supplied error code. */ char* Http2ErrorString (int error) { /*********/ /* begin */ /*********/ if (error > HTTP2_ERROR_COUNT) return ("*UNKNOWN*"); return (Http2ErrorArray[error]); } /*****************************************************************************/ /* Scan the HTTP/2 connection list for connections that have been idle for the timeout period and close them down elegantly. */ BOOL Http2Supervisor () { HTTP2_STRUCT *h2ptr, *next2; /*********/ /* begin */ /*********/ /** if (WATCH_MODULE(WATCH_MOD_HTTP2)) WatchThis (WATCHALL, WATCH_MOD_HTTP2, "Http2Supervisor()"); **/ /* process the connection list from least to most recent */ for (h2ptr = LIST_GET_HEAD(&Http2List); h2ptr != NULL; h2ptr = next2) { /* get (any) next in list in case the current connection is closed */ next2 = LIST_GET_NEXT(h2ptr); /* mitigate CVE-2023-44487 (after a minimum threshhold) */ if (h2ptr->ServerMaxConcStreams > HTTP2_RST_STREAM_MIN) { /* if there are more than half the allowed concurrent requests */ if (h2ptr->RequestCurrent > h2ptr->ServerMaxConcStreams / 2) { /* and the count of RST_STREAM exceed the max this last second */ if (h2ptr->RstStreamCount > h2ptr->ServerMaxConcStreams) { /* consider it bad behaviour and just close the connection */ if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "RST_STREAM mitigation !AZ !UL/!UL/!UL", h2ptr->NetIoPtr->ClientPtr->IpAddressString, h2ptr->RequestCurrent, h2ptr->ServerMaxConcStreams, h2ptr->RstStreamCount); ErrorNoticed (NULL, 0, "RST_STREAM mitigation !AZ !UL/!UL/!UL", FI_LI, h2ptr->NetIoPtr->ClientPtr->IpAddressString, h2ptr->RequestCurrent, h2ptr->ServerMaxConcStreams, h2ptr->RstStreamCount); Http2ConnectClose (h2ptr); continue; } } } /* reset the per-second counter */ h2ptr->RstStreamCount = 0; if (LIST_GET_COUNT(&h2ptr->StreamList)) { Http2StreamPurge (h2ptr); Http2StreamFlow (h2ptr); } if (LIST_GET_COUNT(&h2ptr->StreamList)) h2ptr->IdleTickSecond = 0; else if (!h2ptr->IdleTickSecond) h2ptr->IdleTickSecond = HttpdTickSecond + Http2IdleSeconds; if (h2ptr->FlowStallTickSecond && h2ptr->FlowStallTickSecond < HttpdTickSecond) { if (Http2FlowCheck (h2ptr, NULL, NULL) < 0) Http2ConnectClose (h2ptr); continue; } if (h2ptr->IdleTickSecond && h2ptr->IdleTickSecond < HttpdTickSecond) { if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "TIMEOUT idle"); Http2ConnectClose (h2ptr); continue; } if (h2ptr->GoAwayTickSecond && h2ptr->GoAwayTickSecond < HttpdTickSecond) { if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "TIMEOUT goaway"); Http2ConnectClose (h2ptr); continue; } if (h2ptr->NetIoPtr->VmsStatus) { /* with each call check if the connection can be closed */ Http2ConnectClose (h2ptr); continue; } if (h2ptr->GoAwayLastStreamIdent) { /* connection has been told to goaway */ Http2ConnectClose (h2ptr); continue; } if (h2ptr->PingBackTickSecond && h2ptr->PingBackTickSecond < HttpdTickSecond) { if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "TIMEOUT ping"); h2ptr->PingBackTickSecond = 0; } else if (Http2PingSeconds && (h2ptr->PingSendTickSecond == 0 || h2ptr->PingSendTickSecond < HttpdTickSecond)) { /* send a ping every so-many seconds */ if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "TIME to ping"); Http2Ping (h2ptr, 0, 0, NULL, 0); h2ptr->PingSendTickSecond = HttpdTickSecond + Http2PingSeconds; h2ptr->PingSendTickSecond = HttpdTickSecond + 600; } /* reset DoS limit counters */ h2ptr->EmptyFrameLimitCount = h2ptr->PingLimitCount = h2ptr->SettingsLimitCount = h2ptr->StreamResetLimitCount = h2ptr->ZeroHeaderLimitCount = 0; } return (LIST_NOT_EMPTY(&Http2List)); } /*****************************************************************************/ /* Check whether the current stream and connection windows will support the supplied write data size (if supplied). If write not specified then just check any stalled stream and connection. If the logical name WASD_HTTP2_FLOW_CHECK is defined then issue a noticed against all non-continue events. The default behaviour (0) to *immediately* drop the stream/connection (return -1) upon flow-control stall. This hopefully prevents a connection stall which many agents seem to experience. Define the logical name to 1 to immediately kludge the window (experimental) by the outstanding data length, hopefully nudging the stream along. Define the logical name to 2 to ignore the window size and send anyway. Define to a number at least 5 to drop the stream/connection after that many seconds stalled without window update. Return -1 to drop the stream or connection, positive number to continue with the write, zero to defer the write. */ int Http2FlowCheck ( HTTP2_STRUCT *h2ptr, HTTP2_STREAM_STRUCT *s2ptr, HTTP2_WRITE_STRUCT *w2ptr ) { static int FlowCheck, FlowCheck2 = -1, TickSecond; static char *fcptr; int dlen; /*********/ /* begin */ /*********/ if (TickSecond < HttpdTickSecond) { /* only check logical a maximum of once every fifteen seconds */ TickSecond = HttpdTickSecond + 15; /* default wait for 10 seconds for window update */ FlowCheck = HTTP2_FLOW_CHECK_SECONDS; if (fcptr = SysTrnLnm (WASD_HTTP2_FLOW_CHECK)) { FlowCheck = abs(atoi(fcptr)); if (FlowCheck > 60) FlowCheck = 60; if (FlowCheck != FlowCheck2) { ErrorNoticed (NULL, 0, "WASD_HTTP2_FLOW_CHECK !SL (!SL)", FI_LI, FlowCheck, FlowCheck2); FlowCheck2 = FlowCheck; } } } /* if evaluating an actual write datagram, otherwise checking stall */ if (w2ptr) dlen = w2ptr->DataLength; else dlen = 0; /* always must be a HTTP/2 pointer but stream pointer is optional */ if (s2ptr) { if (dlen) { if (FlowCheck == 0) { if (dlen > s2ptr->WriteWindowSize) { /* immediately drop the stream */ h2ptr->FlowFrameCount++; h2ptr->FlowFrameTally++; if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "STALL stream !UL DROP", s2ptr->Ident); if (fcptr) Http2FlowStallNoticed (FI_LI, h2ptr, s2ptr, dlen); /* this sentinal value is used to trigger a stream drop */ s2ptr->FlowStallTickSecond = HTTP2_STREAM_DROP_NOW; return (-1); } } if (FlowCheck == 1) { if (dlen > s2ptr->WriteWindowSize) { /* kludge the window size by the required data size */ if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "STALL stream !UL KLUDGE !SL + !UL", s2ptr->Ident, s2ptr->WriteWindowSize, dlen); if (fcptr) Http2FlowStallNoticed (FI_LI, h2ptr, s2ptr, dlen); s2ptr->WriteWindowSize += dlen; h2ptr->WriteWindowSize += dlen; h2ptr->FlowFrameCount++; h2ptr->FlowFrameTally++; } /* continue the stream */ return (1); } if (FlowCheck == 2) { if (dlen > s2ptr->WriteWindowSize) { if (fcptr) Http2FlowStallNoticed (FI_LI, h2ptr, s2ptr, dlen); h2ptr->FlowFrameCount++; h2ptr->FlowFrameTally++; if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "STALL stream !UL IGNORE", s2ptr->Ident); } /* continue the stream */ return (1); } /* drop through to handle stalls */ } if (s2ptr->FlowStallTickSecond) { /* stream already stalled */ if (dlen && dlen <= s2ptr->WriteWindowSize) { /* now available flow continue the write */ s2ptr->FlowStallTickSecond = 0; if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "UNSTALL stream !UL", s2ptr->Ident); if (fcptr) Http2FlowStallNoticed (FI_LI, h2ptr, s2ptr, dlen); return (1); } if (s2ptr->FlowStallTickSecond > HTTP2_STREAM_DROPPED && s2ptr->FlowStallTickSecond < HttpdTickSecond) { /* expired, drop the stream */ if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "STALL stream !UL EXPIRED", s2ptr->Ident); if (fcptr) Http2FlowStallNoticed (FI_LI, NULL, s2ptr, dlen); return (-1); } /* stall the write */ return (0); } else if (dlen && dlen > s2ptr->WriteWindowSize) { /* stall the stream for this many seconds */ s2ptr->FlowStallTickSecond = HttpdTickSecond + FlowCheck; h2ptr->FlowFrameCount++; h2ptr->FlowFrameTally++; if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "STALL stream !UL !UL secs !SL > !SL", s2ptr->Ident, FlowCheck, dlen, s2ptr->WriteWindowSize); if (fcptr) Http2FlowStallNoticed (FI_LI, NULL, s2ptr, dlen); return (0); } /* continue the write */ return (1); } if (dlen) { if (FlowCheck == 0) { if (dlen > h2ptr->WriteWindowSize) { /* immediately drop the connection */ h2ptr->FlowStallCount++; h2ptr->FlowStallTally++; if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "STALL connection DROP"); if (fcptr) Http2FlowStallNoticed (FI_LI, h2ptr, s2ptr, dlen); /* this sentinal value is used to trigger a connection drop */ h2ptr->FlowStallTickSecond = HTTP2_STREAM_DROP_NOW; return (-1); } } if (FlowCheck == 1) { if (dlen > h2ptr->WriteWindowSize) { /* kludge the window size by the required data size */ if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "STALL connection KLUDGE !SL + !UL", h2ptr->WriteWindowSize, dlen); if (fcptr) Http2FlowStallNoticed (FI_LI, h2ptr, s2ptr, dlen); s2ptr->WriteWindowSize += dlen; h2ptr->WriteWindowSize += dlen; h2ptr->FlowStallCount++; h2ptr->FlowStallTally++; } /* continue the stream */ return (1); } if (FlowCheck == 2) { if (dlen > h2ptr->WriteWindowSize) { if (fcptr) Http2FlowStallNoticed (FI_LI, h2ptr, s2ptr, dlen); h2ptr->FlowFrameCount++; h2ptr->FlowFrameTally++; if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "STALL connection IGNORE"); } /* continue the stream */ return (1); } /* drop through to handle stalls */ } if (h2ptr->FlowStallTickSecond) { /* connection already stalled */ if (dlen && dlen <= h2ptr->WriteWindowSize) { /* now available flow continue the write */ h2ptr->FlowStallTickSecond = 0; if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "UNSTALL connection"); if (fcptr) Http2FlowStallNoticed (FI_LI, h2ptr, s2ptr, dlen); return (1); } if (h2ptr->FlowStallTickSecond > HTTP2_STREAM_DROPPED && h2ptr->FlowStallTickSecond < HttpdTickSecond) { /* drop the stream */ h2ptr->FlowStallCount++; h2ptr->FlowStallTally++; if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "STALL connection EXPIRED"); if (fcptr) Http2FlowStallNoticed (FI_LI, h2ptr, s2ptr, dlen); return (-1); } } else if (dlen && dlen > h2ptr->WriteWindowSize) { /* stall the stream for this many seconds */ s2ptr->FlowStallTickSecond = HttpdTickSecond + FlowCheck; h2ptr->FlowStallCount++; h2ptr->FlowStallTally++; if (WATCHING (h2ptr, WATCH_HTTP2)) WatchThis (WATCHITM(h2ptr), WATCH_HTTP2, "STALL connection !UL secs !SL > !SL", FlowCheck, dlen, h2ptr->WriteWindowSize); if (fcptr) Http2FlowStallNoticed (FI_LI, h2ptr, s2ptr, dlen); return (0); } /* continue the write */ return (1); } /*****************************************************************************/ /* When logical name WASD_HTTP2_FLOW_CHECK defined report an adverse flow check. */ void Http2FlowStallNoticed ( char *SourceModule, int SourceLine, HTTP2_STRUCT *h2ptr, HTTP2_STREAM_STRUCT *s2ptr, int dlen ) { /*********/ /* begin */ /*********/ if (WATCH_MODULE(WATCH_MOD_HTTP2)) WatchThis (WATCHALL, WATCH_MOD_HTTP2, "Http2FlowStallNoticed() !AZ:!UL 0x!8XL 0x!8XL !UL", SourceModule, SourceLine, h2ptr, s2ptr, dlen); if (s2ptr) { /* this sentinal value is used to trigger a stream drop */ if (s2ptr->FlowStallTickSecond == 1) return; if (!h2ptr) h2ptr = s2ptr->Http2Ptr; if (s2ptr->RequestPtr) { ErrorNoticed (NULL, 0, "!AZ stream !UL(!UL) window:!SL data:!UL tick:!UL:!UL", SourceModule, SourceLine, (s2ptr->FlowStallTickSecond || h2ptr->FlowStallTickSecond) ? "STALL" : "FLOW", s2ptr->RequestPtr->ConnectNumber, h2ptr->ConnectNumber, s2ptr->WriteWindowSize, dlen, s2ptr->FlowStallTickSecond, h2ptr->FlowStallTickSecond); FaoToStdout ("-NOTICED-I-URI, !AZ\n", s2ptr->RequestPtr->rqHeader.RequestUriPtr); FaoToStdout ("-NOTICED-I-AGENT, !AZ\n", s2ptr->RequestPtr->rqHeader.UserAgentPtr); } else ErrorNoticed (NULL, 0, "!AZ stream (!UL) window:!SL data:!UL tick:!UL:!UL", SourceModule, SourceLine, (s2ptr->FlowStallTickSecond || h2ptr->FlowStallTickSecond) ? "STALL" : "FLOW", h2ptr->ConnectNumber, s2ptr->WriteWindowSize, dlen, s2ptr->FlowStallTickSecond, h2ptr->FlowStallTickSecond); } else if (h2ptr) { /* this sentinal value is used to trigger a connection drop */ if (h2ptr->FlowStallTickSecond == 1) return; ErrorNoticed (NULL, 0, "!AZ connection (!UL) window:!SL data:!UL tick:!UL", SourceModule, SourceLine, h2ptr->FlowStallTickSecond ? "STALL" : "FLOW", h2ptr->ConnectNumber, h2ptr->WriteWindowSize, dlen, h2ptr->FlowStallTickSecond); } else ErrorNoticed (NULL, SS$_BUGCHECK, NULL, SourceModule, SourceLine); FaoToStdout ("-NOTICED-I-SERVICE, !AZ//!AZ\n", h2ptr->ServicePtr->RequestSchemeNamePtr, h2ptr->ServicePtr->ServerHostPort); } /*****************************************************************************/ /* Zero HTTP/2 accounting. See items in Http2Report(). */ void Http2ZeroAccounting () { /*********/ /* begin */ /*********/ if (WATCH_MODULE(WATCH_MOD_HTTP2)) WatchThis (WATCHALL, WATCH_MOD_HTTP2, "Http2ZeroAccounting() !UL !&B", InstanceNodeConfig, InstanceNodeSupervisor); /* in a multi-instance config, shared data only by the supervisor */ if (InstanceNodeConfig > 1 && !InstanceNodeSupervisor) return; InstanceMutexLock (INSTANCE_MUTEX_HTTPD); acptr->RequestHttp2Count = 0; acptr->ProcessingPeak[HTTP2] = 0; acptr->BytesPerSecondRawRx64[HTTP2] = 0; acptr->BytesPerSecondRawTotal64[HTTP2] = 0; acptr->BytesPerSecondRawTx64[HTTP2] = 0; acptr->BytesPerSecondRawTotal64[HTTP2] = 0; acptr->BytesPerSecondMax[HTTP2] = 0; acptr->BytesPerSecondMaxDuration64[HTTP2] = 0; acptr->BytesPerSecondMaxBytes64[HTTP2] = 0; acptr->BytesPerSecondMin[HTTP2] = 0; acptr->BytesPerSecondMinDuration64[HTTP2] = 0; acptr->BytesPerSecondMinBytes64[HTTP2] = 0; acptr->ResponseDurationMin64[HTTP2] = 0; acptr->ResponseDurationMax64[HTTP2] = 0; acptr->ResponseDuration64[HTTP2] = 0; acptr->BytesPerSecondAve[HTTP2] = 0; acptr->BytesPerSecondMin[HTTP2] = 0; acptr->BytesPerSecondMax[HTTP2] = 0; acptr->Http2FrameCountRx64 = 0; acptr->Http2FrameCountTx64 = 0; acptr->Http2FrameRequestCountRx64 = 0; acptr->Http2FrameRequestCountTx64 = 0; acptr->Http2FlowFrameCount64 = 0; acptr->Http2FlowStallCount64 = 0; acptr->Http2FrameCountRx64 = 0; acptr->Http2FrameCountTx64 = 0; InstanceMutexUnLock (INSTANCE_MUTEX_HTTPD); } /*****************************************************************************/ /* */ void Http2Report (REQUEST_STRUCT *rqptr) { static char PageBeginFao [] = "

\n\ \n\
\n\ \n\ \ \ \ \ \n\ \ \ \ \ \n\ \ \ \n\ \n\ \n\ \ \ \n\ \n\ \n\ \ \ \n\ \n\ \n\ \ \n\ \n\ \n\ \n\ \n\ \n\ \n\ \
HTTP/2HTTP/1.n
Requests  /Total:!&L(!UL%)!&L(!UL%)
/Peak:!&L!&L
Bytes  /Min:!&,@UQ!&,@UQ
/Max:!&,@UQ!&,@UQ
/Ave:!&L!&L
Duration  /Min:!AZ!AZ
/Max:!AZ!AZ
/Ave:!AZ!AZ
Bytes/Sec  /Min:!&L!&L
/Max:!&L!&L
/Ave:!&L!&L
\ Frames  /Total:!&,@UQ
/Request:!&,@UQ(!UL%)
/Average:!&L
/Flow:!&,@UQ(!UL%)
/Stall:!&,@UQ(!UL%)
/Rx:!&,@UQ
/Tx:!&,@UQ
\n\
\n"; /* the final column just adds a little white-space on the page far right */ static char Http2TableFao [] = "

\n\ \ \ \ \ \ \ \n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \n\ \n"; /* the empty 99% column just forces the rest left */ static char Http2EntryFao [] = "\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \n"; static char Http2EmptyFao [] = "\ \n"; static char PageEndFao [] = "
FramesBytesWrite QueueHPACKRequests
ServiceClientDurationRxTxFloStlRxTx0HNLPkRxTxTotPkCurWATCH
!3ZL!AZ//!AZ!AZ,!UL!AZ!&L!&L!UL!UL!&,@UQ!&,@UQ!UL!UL!UL!UL!UL!UL%!UL%!&L!UL!UL!&@
000empty
\n\

\n\ \n\
\n\
\n\ \n\
\n\ 		\n\
\n\ \n\
\n\
\n\ !AZ\ \n\ \n\ \n"; int idx, status, EntryCount, RequestHttpCount, RequestHttp1Count, RequestHttp2Count; int64 bytes64, Time64, ConnectTime64, FrameRequest64, FrameTotal64; ulong FaoVector [48]; ulong *vecptr; char *cptr; HTTP2_STRUCT *h2ptr; /*********/ /* begin */ /*********/ if (WATCHMOD (rqptr, WATCH_MOD_HTTP2)) WatchThis (WATCHITM(rqptr), WATCH_MOD_HTTP2, "Http2Report()"); AdminPageTitle (rqptr, "HTTP Report"); vecptr = FaoVector; InstanceMutexLock (INSTANCE_MUTEX_HTTPD); RequestHttp1Count = acptr->RequestHttp11Count + acptr->RequestHttp10Count; RequestHttp1Count -= acptr->DoProxyCount; RequestHttp2Count = acptr->RequestHttp2Count; RequestHttpCount = RequestHttp1Count + RequestHttp2Count; *vecptr++ = RequestHttp2Count; *vecptr++ = PercentOf32 (RequestHttp2Count, RequestHttpCount); *vecptr++ = RequestHttp1Count; *vecptr++ = PercentOf32 (RequestHttp1Count, RequestHttpCount); *vecptr++ = acptr->ProcessingPeak[HTTP2]; *vecptr++ = acptr->ProcessingPeak[HTTP1]; *vecptr++ = &acptr->BytesPerSecondMinBytes64[HTTP2]; *vecptr++ = &acptr->BytesPerSecondMinBytes64[HTTP1]; *vecptr++ = &acptr->BytesPerSecondMaxBytes64[HTTP2]; *vecptr++ = &acptr->BytesPerSecondMaxBytes64[HTTP1]; if (acptr->RequestHttp2Count) *vecptr++ = (ulong)(float)acptr->BytesPerSecondRawTotal64[HTTP2] / (float)acptr->RequestHttp2Count; else *vecptr++ = 0; if (acptr->RequestHttp11Count + acptr->RequestHttp10Count) *vecptr++ = (ulong)(float)acptr->BytesPerSecondRawTotal64[HTTP1] / (float)(acptr->RequestHttp11Count + acptr->RequestHttp10Count); else *vecptr++ = 0; bytes64 = acptr->ResponseDurationMin64[HTTP2]; *vecptr++ = DurationString (rqptr, &bytes64); bytes64 = acptr->ResponseDurationMin64[HTTP1]; *vecptr++ = DurationString (rqptr, &bytes64); bytes64 = acptr->ResponseDurationMax64[HTTP2]; *vecptr++ = DurationString (rqptr, &bytes64); bytes64 = acptr->ResponseDurationMax64[HTTP1]; *vecptr++ = DurationString (rqptr, &bytes64); bytes64 = acptr->ResponseDuration64[HTTP2]; *vecptr++ = AverageDurationString (rqptr, &bytes64, acptr->ResponseDurationCount[HTTP2]); bytes64 = acptr->ResponseDuration64[HTTP1]; *vecptr++ = AverageDurationString (rqptr, &bytes64, acptr->ResponseDurationCount[HTTP1]); *vecptr++ = acptr->BytesPerSecondMin[HTTP2]; *vecptr++ = acptr->BytesPerSecondMin[HTTP1]; *vecptr++ = acptr->BytesPerSecondMax[HTTP2]; *vecptr++ = acptr->BytesPerSecondMax[HTTP1]; *vecptr++ = acptr->BytesPerSecondAve[HTTP2]; *vecptr++ = acptr->BytesPerSecondAve[HTTP1]; FrameTotal64 = acptr->Http2FrameCountRx64; FrameTotal64 += acptr->Http2FrameCountTx64; *vecptr++ = &FrameTotal64; FrameRequest64 = acptr->Http2FrameRequestCountRx64; FrameRequest64 += acptr->Http2FrameRequestCountTx64; *vecptr++ = &FrameRequest64; *vecptr++ = PercentOf64 (&FrameRequest64, &FrameTotal64); if (acptr->RequestHttp2Count) *vecptr++ = (ulong)((float)FrameTotal64 / (float)acptr->RequestHttp2Count); else *vecptr++ = 0; *vecptr++ = &acptr->Http2FlowFrameCount64; *vecptr++ = PercentOf64 (&acptr->Http2FlowFrameCount64, &acptr->Http2FrameRequestCountTx64); *vecptr++ = &acptr->Http2FlowStallCount64; *vecptr++ = PercentOf64 (&acptr->Http2FlowStallCount64, &acptr->Http2FrameRequestCountTx64); *vecptr++ = &acptr->Http2FrameCountRx64; *vecptr++ = &acptr->Http2FrameCountTx64; InstanceMutexUnLock (INSTANCE_MUTEX_HTTPD); FaoCheck (sizeof(FaoVector), &FaoVector, vecptr, FI_LI); status = FaolToNet (rqptr, PageBeginFao, &FaoVector); if (VMSnok (status)) ErrorNoticed (rqptr, status, NULL, FI_LI); /***********************/ /* HTTP/2 list entries */ /***********************/ status = FaolToNet (rqptr, Http2TableFao, NULL); if (VMSnok (status)) ErrorNoticed (rqptr, status, NULL, FI_LI); EntryCount = 0; sys$gettim (&Time64); /* process the request list from least to most recent */ for (h2ptr = LIST_GET_HEAD(&Http2List); h2ptr != NULL; h2ptr = LIST_GET_NEXT(h2ptr)) { EntryCount++; vecptr = FaoVector; ConnectTime64 = h2ptr->ConnectTime64 - Time64; *vecptr++ = EntryCount % 2 ? " class=\"hlght\"" : ""; *vecptr++ = EntryCount; *vecptr++ = h2ptr->ServicePtr->RequestSchemeNamePtr; *vecptr++ = h2ptr->ServicePtr->ServerHostPort; *vecptr++ = h2ptr->ClientPtr->Lookup.HostName; *vecptr++ = h2ptr->ClientPtr->IpPort; *vecptr++ = DurationString (rqptr, &ConnectTime64); *vecptr++ = h2ptr->FrameCountRx; *vecptr++ = h2ptr->FrameCountTx; *vecptr++ = h2ptr->FlowFrameCount; *vecptr++ = h2ptr->FlowStallCount; *vecptr++ = &h2ptr->BytesRawRx64; *vecptr++ = &h2ptr->BytesRawTx64; for (idx = HTTP2_WRITE_QUEUE_MAX; idx <= HTTP2_WRITE_QUEUE_LOW; idx++) *vecptr++ = LIST_GET_COUNT (&h2ptr->QueuedWriteList[idx]); *vecptr++ = h2ptr->QueuedWritePeak; *vecptr++ = ADMIN_REPORT_HPACK; *vecptr++ = h2ptr->ConnectNumber; if (h2ptr->HpackClientOutputCount) *vecptr++ = (h2ptr->HpackClientInputCount * 100) / h2ptr->HpackClientOutputCount; else *vecptr++ = 0; *vecptr++ = ADMIN_REPORT_HPACK; *vecptr++ = h2ptr->ConnectNumber; if (h2ptr->HpackServerInputCount) *vecptr++ = (h2ptr->HpackServerOutputCount * 100) / h2ptr->HpackServerInputCount; else *vecptr++ = 0; *vecptr++ = h2ptr->RequestCount; *vecptr++ = h2ptr->RequestPeak; *vecptr++ = h2ptr->RequestCurrent; if (rqptr->Http2Ptr && rqptr->Http2Ptr->ConnectNumber == h2ptr->ConnectNumber) *vecptr++ = "current"; else { *vecptr++ = "P\ +\ W"; *vecptr++ = ADMIN_REPORT_WATCH; *vecptr++ = h2ptr->ConnectNumber; *vecptr++ = ADMIN_REPORT_WATCH; *vecptr++ = h2ptr->ConnectNumber; *vecptr++ = h2ptr->ConnectNumber; *vecptr++ = ADMIN_REPORT_WATCH; *vecptr++ = h2ptr->ConnectNumber; } FaoCheck (sizeof(FaoVector), &FaoVector, vecptr, FI_LI); status = FaolToNet (rqptr, Http2EntryFao, &FaoVector); if (VMSnok (status)) ErrorNoticed (rqptr, status, NULL, FI_LI); } if (!EntryCount) FaolToNet (rqptr, Http2EmptyFao, NULL); /**************/ /* end report */ /**************/ vecptr = FaoVector; *vecptr++ = ADMIN_CONTROL_HTTP2_PURGE; *vecptr++ = ADMIN_CONTROL_HTTP2_PURGE_ALL, *vecptr++ = AdminRefresh (rqptr); FaolToNet (rqptr, PageEndFao, FaoVector); rqptr->rqResponse.PreExpired = PRE_EXPIRE_ADMIN; ResponseHeader200 (rqptr, "text/html", NULL); AdminEnd (rqptr); } /*****************************************************************************/ /* See NetTestBreak(). */ #if WATCH_MOD void Http2TestBreak (int every) { int count, total; HTTP2_STRUCT *h2ptr; HTTP2_STREAM_STRUCT *s2ptr; NETIO_STRUCT *ioptr; REQUEST_STRUCT *rqeptr; /*********/ /* begin */ /*********/ if (every < 0) every = 1; count = total = 0; for (h2ptr = LIST_GET_HEAD(&Http2List); h2ptr != NULL; h2ptr = LIST_GET_NEXT(h2ptr)) { if (++count % every) continue; ioptr = h2ptr->NetIoPtr; if (ioptr->Channel) { /* do not kill WATCHing request */ rqeptr = (REQUEST_STRUCT*)-1; for (s2ptr = LIST_GET_HEAD(&h2ptr->StreamList); s2ptr != NULL; s2ptr = LIST_GET_NEXT(s2ptr)) { rqeptr = (REQUEST_STRUCT*)s2ptr->RequestPtr; if (rqeptr == Watch.RequestPtr) break; } if (rqeptr == Watch.RequestPtr) continue; WatchThis (WATCHITM(h2ptr), WATCH_NETWORK, "BREAK2 !AZ", h2ptr->ServicePtr->ServerHostPort); sys$dassgn (ioptr->Channel); ioptr->Channel = 0; total++; for (s2ptr = LIST_GET_HEAD(&h2ptr->StreamList); s2ptr != NULL; s2ptr = LIST_GET_NEXT(s2ptr)) { rqeptr = (REQUEST_STRUCT*)s2ptr->RequestPtr; if (rqeptr->rqHeader.RequestUriPtr) WatchThis (WATCHITM(rqeptr), WATCH_NETWORK, "BREAK2 !AZ !AZ !AZ", rqeptr->ServicePtr->ServerHostPort, rqeptr->rqHeader.MethodName, rqeptr->rqHeader.RequestUriPtr); else WatchThis (WATCHITM(rqeptr), WATCH_NETWORK, "BREAK2 !AZ 0", rqeptr->ServicePtr->ServerHostPort); } } } FaoToStdout ("%HTTPD-I-HTTP2, !%T, !UL/!UL/!UL broken\n", 0, every, total, count); } #endif /* WATCH_MOD */ /*****************************************************************************/