Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTTPCLIENT-2350 - Enhance HttpClientConnectionOperator with Optional DNS Resolution Control #598

Merged
merged 1 commit into from
Nov 16, 2024

Conversation

arturobernalg
Copy link
Member

This PR introduces an enhancement to DefaultHttpClientConnectionOperator, adding the ability to disable DNS resolution for specific connections. This is particularly useful when connecting to .onion addresses through a SOCKS proxy, where public DNS lookups are not possible or desirable.

@arturobernalg arturobernalg requested a review from ok2c November 14, 2024 09:23
@arturobernalg arturobernalg changed the title Enhance HttpClientConnectionOperator with Optional DNS Resolution Control HTTPCLIENT-2350 - Enhance HttpClientConnectionOperator with Optional DNS Resolution Control Nov 14, 2024
@ok2c
Copy link
Member

ok2c commented Nov 14, 2024

@arturobernalg Damn. I also started working on this one. Conceptually I am not in favor of adding a new config nob every time someone asks for it. We did that often in the days of HC 3 and did us little good. I would prefer this issue solved with a custom Function<HttpHost, InetSocketAddress[]> address resolver or by making DnsResolver able to return unresolved addresses.

@@ -153,43 +153,50 @@ public void connect(
final Timeout connectTimeout,
final SocketConfig socketConfig,
final Object attachment,
final HttpContext context) throws IOException {
final HttpContext context,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be a lot more flexible if this could be configured through the context (that's what we did to this forked class).


if (disableDnsResolution) {
// Hostname is an .onion address. Do not try to resolve it to an InetAddress.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or not, it's not determining what the address is.

@thc202
Copy link

thc202 commented Nov 14, 2024

I would prefer this issue solved with a custom Function<HttpHost, InetSocketAddress[]> address resolver or by making DnsResolver able to return unresolved addresses.

I'd much be in favour of something like that.

@arturobernalg
Copy link
Member Author

@arturobernalg Damn. I also started working on this one. Conceptually I am not in favor of adding a new config nob every time someone asks for it. We did that often in the days of HC 3 and did us little good. I would prefer this issue solved with a custom Function<HttpHost, InetSocketAddress[]> address resolver or by making DnsResolver able to return unresolved addresses.

Fair enough, @ok2c . I agree with your point about avoiding adding new configuration knobs unnecessarily. I've updated the implementation to use a Function. Please take another pass

@arturobernalg arturobernalg requested a review from thc202 November 14, 2024 12:23
@ok2c
Copy link
Member

ok2c commented Nov 14, 2024

@arturobernalg My idea was to add this method to DnsResolver

    /**
     * Returns a list of {@link SocketAddress} for the given host with the given port.
     *
     * @see SocketAddress
     *
     * @since 5.5
     */
    default List<SocketAddress> resolve(String host, int port) throws UnknownHostException {
        InetAddress[] inetAddresses = resolve(host);
        if (inetAddresses == null) {
            return Collections.singletonList(InetSocketAddress.createUnresolved(host, port));
        }
        return Arrays.stream(inetAddresses)
                .map(e -> new InetSocketAddress(e, port))
                .collect(Collectors.toList());
    }

@arturobernalg
Copy link
Member Author

@arturobernalg My idea was to add this method to DnsResolver

    /**
     * Returns a list of {@link SocketAddress} for the given host with the given port.
     *
     * @see SocketAddress
     *
     * @since 5.5
     */
    default List<SocketAddress> resolve(String host, int port) throws UnknownHostException {
        InetAddress[] inetAddresses = resolve(host);
        if (inetAddresses == null) {
            return Collections.singletonList(InetSocketAddress.createUnresolved(host, port));
        }
        return Arrays.stream(inetAddresses)
                .map(e -> new InetSocketAddress(e, port))
                .collect(Collectors.toList());
    }

What about now, @ok2c ? I believe I've made the correct changes this time. Please review.

Copy link
Member

@ok2c ok2c left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arturobernalg Looks good to me, but similar change needs to be doe to MultihomeIOSessionRequester as well.

Mockito.eq(null),
Mockito.any(),
Mockito.eq(null)))
Mockito.eq(route),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arturobernalg Can we avoid formatting changes in the same commit with functional ones? If you want to change formatting please do it through the style check or at least in a separate commit.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pelase @ok2c do another pass.

@arturobernalg arturobernalg force-pushed the HTTPCLIENT-2350 branch 4 times, most recently from 84d0099 to 7b678ce Compare November 15, 2024 20:23
@arturobernalg arturobernalg requested a review from ok2c November 15, 2024 20:33
*
* @since 5.5
*/
default List<SocketAddress> resolve(String host, int port) throws UnknownHostException {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arturobernalg Almost there. Let's change this method to return List<InetSocketAddress> in order to avoid ugly downcasts in several places.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@arturobernalg arturobernalg force-pushed the HTTPCLIENT-2350 branch 2 times, most recently from dee610f to 17395ad Compare November 16, 2024 12:52
@arturobernalg arturobernalg requested a review from ok2c November 16, 2024 12:59
@@ -129,7 +129,7 @@ public void cancelled() {

void executeNext() {
final int index = attempt.getAndIncrement();
final InetSocketAddress remoteAddress = new InetSocketAddress(remoteAddresses[index], remoteEndpoint.getPort());
final InetSocketAddress remoteAddress = (InetSocketAddress) remoteAddresses.get(index);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arturobernalg I suppose this cast is no longer needed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ok2c I don't know what happened. I just removed.

…onnectionOperator to enhance flexibility in address resolution, specifically allowing for direct handling of unresolved addresses. Updated DnsResolver to introduce a new resolve method supporting both standard and bypassed DNS lookups, enabling improved support for non-public resolvable hosts like .onion endpoints via SOCKS proxy. Adjusted related tests to align with the new resolution mechanism.
@arturobernalg arturobernalg merged commit 0b56a62 into apache:master Nov 16, 2024
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants