Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fwv #1

Draft
wants to merge 32 commits into
base: main
Choose a base branch
from
Draft
Changes from 1 commit
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
a33cb3d
fwv
jhoblitt Nov 12, 2024
9ea211e
one pool
jhoblitt Nov 12, 2024
8321484
disable file slurping
jhoblitt Nov 12, 2024
ba454a0
add S3DAEMON_MAX_PARALLEL_UPLOADS
jhoblitt Nov 12, 2024
b99de7d
add S3DAEMON_UPLOAD_TIMEOUT
jhoblitt Nov 12, 2024
7830ec1
add basic .gitignore
jhoblitt Nov 12, 2024
2618514
add Dockerfile
jhoblitt Nov 12, 2024
0822c16
set 5MiB http WriteBufferSize
jhoblitt Nov 12, 2024
fb8e933
rm unused S3DHandler.UploadFile
jhoblitt Nov 12, 2024
fc898b1
add S3DAEMON_QUEUE_TIMEOUT & S3DAEMON_UPLOAD_TRIES + cleanup logs
jhoblitt Nov 13, 2024
75b6209
switch to github.com/hyperledger/fabric/common/semaphore
jhoblitt Nov 13, 2024
773360f
increase max idle conns to allow for multipart create/complete
jhoblitt Nov 14, 2024
efc17d7
add S3DAEMON_UPLOAD_PARTSIZE
jhoblitt Nov 14, 2024
5abb9dc
decrease write buffer to 64MiB
jhoblitt Nov 18, 2024
97e9350
disable aws retyring (aws.NopRetryer)
jhoblitt Nov 18, 2024
52b7f54
explicitly enable http keep alive
jhoblitt Nov 18, 2024
144fcd2
statically link docker build
jhoblitt Nov 18, 2024
887ade2
add 100mbit/s upload rate limit
jhoblitt Nov 18, 2024
fde8fad
add S3DAEMON_UPLOAD_BWLIMIT
jhoblitt Nov 18, 2024
b1de9a9
pass through client request context
jhoblitt Nov 18, 2024
89113b7
mv http form parsing into parseRequest()
jhoblitt Nov 19, 2024
ee4ac10
stop upload tries when client disconnects
jhoblitt Nov 19, 2024
627a0bc
convert S3DAEMON_UPLOAD_BWLIMIT to bit/s; 0 disables
jhoblitt Nov 19, 2024
1b65e16
add fq bwlimit as socket option and S3DAEMON_UPLOAD_BWLIMIT_INTERNAL
jhoblitt Nov 19, 2024
28120ee
add S3DAEMON_UPLOAD_WRITE_BUFFER_SIZE
jhoblitt Nov 21, 2024
1b2ba23
rm commented out code and stale comments
jhoblitt Nov 22, 2024
70fb4b1
add Makefile for static build
jhoblitt Nov 22, 2024
37021b5
rename s3daemon-go -> s3nd
jhoblitt Nov 22, 2024
09ee251
add gha ci workflow
jhoblitt Nov 22, 2024
06e0b70
rm S3ND_UPLOAD_BWLIMIT_INTERNAL & github.com/conduitio/bwlimit
jhoblitt Nov 22, 2024
3ad74f9
mv env var and flag handling into conf package
jhoblitt Nov 22, 2024
1a94702
mv http.Handler implementation into handler package
jhoblitt Nov 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
decrease write buffer to 64MiB
jhoblitt committed Nov 22, 2024
commit 5abb9dcb248fa9f071faff8eabd3b8fbb01368e5
2 changes: 1 addition & 1 deletion main.go
Original file line number Diff line number Diff line change
@@ -252,7 +252,7 @@ func NewHandler(conf *S3DConf) *S3DHandler {
t.MaxIdleConns = maxConns
t.MaxConnsPerHost = maxConns
t.MaxIdleConnsPerHost = maxConns
t.WriteBufferSize = 1024 * 1024 * 5
t.WriteBufferSize = 64 * 1024
// disable http/2 to prevent muxing over a single tcp connection
t.ForceAttemptHTTP2 = false
t.TLSClientConfig.NextProtos = []string{"http/1.1"}