getBufferedAmount() functionality for HttpResponse<SSL>

[MacroMelon]

Hello! I was wondering if there was a way to get the backpressure in an HttpResponse object.

My implementation is such that I need to be able to wait till the the backpressure has gone to zero to continue sending, but I cant use callbacks (like onWriteable) because I'm executing this in another callback - yes, it's a bit messy but at the moment I dont really have a better solution :(

Context: In the route handler I'm using libcurl to httpget a large file, and I want to write the contents to the HttpResponse as the data comes. I know for a fact that the HttpResponse client transfer speed is slower than the curl get (in my case) and a buildup of backpressure is basically garanteed. I'm writing in the curl write function as follows:

//define curl obj and init up here 

struct writefunContext {
    HttpResponse<true> * cc;
    CURL *curl;
};

writefunContext context = {clientConnection, curl};

curl_easy_setopt(curl, CURLOPT_WRITEDATA, &context);
const bool ssl = true;

curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, +[](void* ptr, size_t size, size_t nmemb, writefunContext* context)-> size_t{
    if (!context->cc->write(std::string((char*)ptr, nmemb))){
        std::cout << "Backpressure detected" << std::endl;
        curl_easy_pause(context->curl, CURLPAUSE_RECV);

        //WAIT HERE TILL BACK PRESSURE 0

        curl_easy_pause(context->curl, CURLPAUSE_CONT);
        std::cout << "Backpressure releived" << std::endl;
    }
    else{
        std::cout << "nb" << std::endl;
    }
    return size * nmemb;
});

setResponseCORSHeaders(clientConnection);
clientConnection->writeStatus("200");

curl_easy_perform(curl);    //do the thing (send the request)

//end transfer and cleanup down here

(dont mind the exessive cout, those are just for debugging, clientConnection is the HttpResponse object)

So it flows like: writeFunction gets called for new data chunk, it writes it to the HttpResponse object, if backpressure it pauses the curl transfer till the backpressure is zero and then resumes it.

If I could pause libcurl as the writefunction exits and unpause in the onWriteable handler it would be perfect, but curl_easy_perform is non-blocking and I have a feeling it is holding the uWS code from running (I'm not sure though). I've also tried switching to a curl_multi system that doesnt block, and waiting in a while loop but the onWritable still doesn't get called.

Here is my onWriteable if interested:

clientConnection->onWritable([curl](uintmax_t a) -> bool {
            std::cout << "Backpressure releived" << std::endl;
            curl_easy_pause(curl, CURLPAUSE_CONT);
            return true;
        });

[uNetworkingAB]

If you use curl_easy_perform then you are using the blocking calls from what I understand, and if you then want to wait for some other thread's signal you want a conditional variable. So whenever your onWritable has getBufferedAmount() 0, then you just signal your conditional variable and then your curl thread will continue.

[uNetworkingAB]

std::condition_variable is a synchronization primitive used with a std::mutex to block one or more threads until another thread both modifies a shared variable (the condition) and notifies the std::condition_variable.

[uNetworkingAB]

But of course, ideally you would use some async library that used the same event-loop library as uWS, such as libuv. Then you would have a seamless integration

[MacroMelon]

Thanks for the quick reply! I'm not using multiple threads, the curl stuff is running on the same thread as the uWS app, so the onWritable function never gets called. I tried to cast the HttpResponse object to an (AsyncSocket *) as I saw that its derived from it, in attempt to try to access the getBufferedAmount() member function so that I can manually keep checking it without relying on the uWS event loop, but that memeber is marked as protected. Is there any other way I can get the buffered amount for the HttpResponse object without the onWriteable callback?

I'll look into libuv but I'm afraid I dont really know much about it and it will take a while to properly figure out and integrate, although as you say that would be a great solution.

[uNetworkingAB]

You need to redesign from scratch because this is not going to work. You can't block the uWS thread and expect getBufferedAmount() to change. Nothing will progress if uWS is blocked.

[MacroMelon]

Thats fair. I took your advice and now I'm using a different approach (I setup the file server as a mounted disk drive and I'm just reading the files normally). That method works, Thank you!
I looked into libuv and it defenitely seems interesting, if my approach hits an unforseen snag I will probably come back to it.
Thanks again!