Skip to content

[HttpClient][Curl] Use poll api to avoid select 1024 FD hard limit#1426

Open
lucianopa-msft wants to merge 1 commit intomicrosoft:mainfrom
lucianopa-msft:lucianopa/use-pool-for-file-descr
Open

[HttpClient][Curl] Use poll api to avoid select 1024 FD hard limit#1426
lucianopa-msft wants to merge 1 commit intomicrosoft:mainfrom
lucianopa-msft:lucianopa/use-pool-for-file-descr

Conversation

@lucianopa-msft
Copy link
Copy Markdown
Member

Using select causes a crash because of its implementation 1024 FD hard limit.

*** bit out of range 0 - FD_SETSIZE on fd_set ***: terminated

This is crash happens depending on the application running if it has held other file descriptors around in the process.
So even though the number of request simultaneously is guarded here:

if (uploadCount() >= static_cast<uint32_t>(m_config[CFG_INT_MAX_PENDING_REQ]) )

It doesn't prevent application to have hold onto other file descriptors opened which is outside of control of this library.

How to repro:

  • Pretend application has done a lot of things that caused them to have a lot of things opened. (We had to pretend in this case because we had not access to customer application nor environment).
  • Then start normal flow of telemetry.
int main () {
 // Pretend application has a lot of file description opened in the current process.
    for (int i = 0; i < 900; i++) {
        int fd = open("/dev/null", O_RDONLY);
        if (fd < 0) {
            break;
        }
    }

    printf("Next fd allocation should be >= 900\n");
   auto handle = my_domain_telemetry_sender_init(...);
   // ..... 
   return 0;
}

TelemetrySenderHandle my_domain_telemetry_sender_init(
    const char* event_collector_uri, const char* ingestion_token) {
  auto& configuration =
      Microsoft::Applications::Events::LogManager::GetLogConfiguration();
  configuration[Microsoft::Applications::Events::CFG_STR_COLLECTOR_URL] =
      event_collector_uri;
  Microsoft::Applications::Events::ILogger* logger =
      Microsoft::Applications::Events::LogManager::Initialize(ingestion_token);
  assert(logger && "Could not create logger");
  return (TelemetrySenderHandle)(new TelemetrySender{logger});
}

Results:

*** bit out of range 0 - FD_SETSIZE on fd_set ***: terminated

Fix

Those changes fix it by not relying on select hard limit and bullet proving our SDK to not fail on conditions that we as SDK do not control such as those and make the library more resilient.

Fixes #1060

@lucianopa-msft lucianopa-msft requested a review from a team as a code owner April 16, 2026 21:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use of select() in lib/http/HttpClient_Curl.hpp causes crashes when >1024 filehandles in use

1 participant