Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RCORE-2155 Store FLX download progress in the Realm file #7780

Merged
merged 1 commit into from
Jun 18, 2024

Conversation

tgoyne
Copy link
Member

@tgoyne tgoyne commented Jun 5, 2024

Multiprocess sync needs this information to be available to non-sync-agent processes, which means storing it in the Realm file like the other progress information. For FLX we previously were always writing 0 to the downloadable_progress field in the history compartment, so we can reuse that field and don't have to make any schema changes. It's an integer field, so the double is stored as a fixed-point value with five digits of precision.

Pending bootstrap bytes similarly need to be read from the Realm file rather than passed directly to SessionWrapper, but that was already being persisted.

The separate notify_upload_progress() and notify_download_progress() functions and only_if_new_uploadable_data flag don't work in a multiprocess context. We need to determine what notifications should be sent based entirely on the previous and current state of the progress information, as that's all that secondary processes have access to. This turned out to not really be a problem.

This should have no functional changes by itself, so no changes to the tests other than updating them to reflect downloadable_bytes no longer being a uint64_t*.

Fixes #7779.

@tgoyne tgoyne self-assigned this Jun 5, 2024
@cla-bot cla-bot bot added the cla: yes label Jun 5, 2024
@tgoyne tgoyne force-pushed the tg/download-progress branch from e77f16b to 71c0158 Compare June 5, 2024 17:45
Copy link

coveralls-official bot commented Jun 5, 2024

Pull Request Test Coverage Report for Build thomas.goyne_394

Details

  • 93 of 95 (97.89%) changed or added relevant lines in 8 files are covered.
  • 73 unchanged lines in 12 files lost coverage.
  • Overall coverage decreased (-0.01%) to 90.862%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/realm/sync/client.cpp 37 38 97.37%
src/realm/sync/noinst/protocol_codec.hpp 4 5 80.0%
Files with Coverage Reduction New Missed Lines %
src/realm/query_expression.hpp 1 93.81%
src/realm/sort_descriptor.cpp 1 94.06%
src/realm/array_blobs_big.cpp 2 98.58%
test/test_lang_bind_helper.cpp 2 93.2%
src/realm/sync/noinst/protocol_codec.hpp 3 74.07%
src/realm/unicode.cpp 3 83.83%
src/realm/link_translator.cpp 4 76.92%
src/realm/sync/instruction_applier.cpp 5 68.01%
src/realm/bplustree.cpp 6 72.55%
src/realm/sync/noinst/client_impl_base.cpp 6 82.36%
Totals Coverage Status
Change from base Build 2387: -0.01%
Covered Lines: 214813
Relevant Lines: 236418

💛 - Coveralls

@tgoyne tgoyne force-pushed the tg/download-progress branch from 71c0158 to 7f03dd0 Compare June 5, 2024 20:37
Copy link

coveralls-official bot commented Jun 5, 2024

Pull Request Test Coverage Report for Build thomas.goyne_395

Details

  • 94 of 96 (97.92%) changed or added relevant lines in 8 files are covered.
  • 46 unchanged lines in 10 files lost coverage.
  • Overall coverage increased (+0.01%) to 90.871%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/realm/sync/client.cpp 37 38 97.37%
src/realm/sync/noinst/protocol_codec.hpp 4 5 80.0%
Files with Coverage Reduction New Missed Lines %
src/realm/sync/instructions.hpp 1 76.03%
src/realm/util/serializer.cpp 1 90.43%
test/test_dictionary.cpp 1 99.83%
src/realm/sync/client.cpp 3 91.74%
src/realm/unicode.cpp 3 83.83%
src/realm/util/future.hpp 3 95.94%
test/fuzz_tester.hpp 4 57.32%
test/fuzz_group.cpp 7 49.76%
src/realm/index_string.cpp 8 84.63%
src/realm/sync/noinst/server/server.cpp 15 73.63%
Totals Coverage Status
Change from base Build 2388: 0.01%
Covered Lines: 214836
Relevant Lines: 236419

💛 - Coveralls

@tgoyne tgoyne marked this pull request as ready for review June 5, 2024 22:03
@tgoyne tgoyne requested review from jbreams and kiburtse June 5, 2024 22:03
@tgoyne tgoyne force-pushed the tg/download-progress branch from 7f03dd0 to 5680af0 Compare June 10, 2024 18:49
Copy link

coveralls-official bot commented Jun 10, 2024

Pull Request Test Coverage Report for Build thomas.goyne_400

Details

  • 94 of 96 (97.92%) changed or added relevant lines in 8 files are covered.
  • 103 unchanged lines in 15 files lost coverage.
  • Overall coverage decreased (-0.001%) to 90.96%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/realm/sync/client.cpp 37 38 97.37%
src/realm/sync/noinst/protocol_codec.hpp 4 5 80.0%
Files with Coverage Reduction New Missed Lines %
src/realm/index_string.hpp 1 93.48%
src/realm/sync/noinst/server/server_history.cpp 1 63.7%
src/realm/uuid.cpp 1 98.48%
test/test_index_string.cpp 1 93.48%
src/realm/mixed.cpp 2 86.46%
test/test_lang_bind_helper.cpp 2 93.2%
src/realm/query_expression.hpp 3 93.81%
src/realm/sync/instruction_applier.cpp 3 68.01%
src/realm/sync/noinst/protocol_codec.hpp 3 74.07%
src/realm/table.cpp 3 90.37%
Totals Coverage Status
Change from base Build 2406: -0.001%
Covered Lines: 214587
Relevant Lines: 235914

💛 - Coveralls

@tgoyne tgoyne force-pushed the tg/download-progress branch from 5680af0 to caff9c2 Compare June 12, 2024 17:21
Copy link

coveralls-official bot commented Jun 12, 2024

Pull Request Test Coverage Report for Build thomas.goyne_412

Details

  • 94 of 96 (97.92%) changed or added relevant lines in 8 files are covered.
  • 73 unchanged lines in 13 files lost coverage.
  • Overall coverage decreased (-0.007%) to 90.955%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/realm/sync/client.cpp 37 38 97.37%
src/realm/sync/noinst/protocol_codec.hpp 4 5 80.0%
Files with Coverage Reduction New Missed Lines %
src/realm/dictionary.cpp 1 85.13%
src/realm/util/compression.cpp 1 89.62%
src/realm/array_blobs_big.cpp 2 98.58%
src/realm/object-store/shared_realm.cpp 2 91.89%
src/realm/query_expression.hpp 2 93.81%
src/realm/table_view.cpp 2 92.99%
src/realm/parser/driver.cpp 3 87.57%
src/realm/link_translator.cpp 4 76.92%
src/realm/bplustree.cpp 6 72.55%
src/realm/sync/network/network.hpp 7 85.28%
Totals Coverage Status
Change from base Build 2408: -0.007%
Covered Lines: 214680
Relevant Lines: 236030

💛 - Coveralls

@tgoyne tgoyne force-pushed the tg/download-progress branch from caff9c2 to 6501c2c Compare June 12, 2024 19:16
Copy link

coveralls-official bot commented Jun 12, 2024

Pull Request Test Coverage Report for Build thomas.goyne_415

Details

  • 94 of 96 (97.92%) changed or added relevant lines in 8 files are covered.
  • 67 unchanged lines in 14 files lost coverage.
  • Overall coverage decreased (-0.007%) to 90.947%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/realm/sync/client.cpp 37 38 97.37%
src/realm/sync/noinst/protocol_codec.hpp 4 5 80.0%
Files with Coverage Reduction New Missed Lines %
src/realm/util/serializer.cpp 1 90.74%
test/test_dictionary.cpp 1 99.83%
test/test_index_string.cpp 1 93.48%
src/realm/collection_parent.cpp 3 93.08%
src/realm/sync/client.cpp 3 91.74%
src/realm/sync/noinst/server/server_history.cpp 3 63.51%
src/realm/table.cpp 3 90.42%
src/realm/alloc_slab.cpp 4 90.56%
src/realm/sync/noinst/client_impl_base.cpp 5 82.51%
test/fuzz_group.cpp 6 47.05%
Totals Coverage Status
Change from base Build 2415: -0.007%
Covered Lines: 214668
Relevant Lines: 236036

💛 - Coveralls

Copy link
Contributor

@jbreams jbreams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat.

@tgoyne tgoyne force-pushed the tg/download-progress branch from 6501c2c to 5931242 Compare June 18, 2024 19:14
Copy link

coveralls-official bot commented Jun 18, 2024

Pull Request Test Coverage Report for Build thomas.goyne_418

Details

  • 94 of 96 (97.92%) changed or added relevant lines in 8 files are covered.
  • 70 unchanged lines in 11 files lost coverage.
  • Overall coverage increased (+0.004%) to 90.973%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/realm/sync/client.cpp 37 38 97.37%
src/realm/sync/noinst/protocol_codec.hpp 4 5 80.0%
Files with Coverage Reduction New Missed Lines %
src/realm/sync/instructions.hpp 1 76.03%
src/realm/sync/noinst/server/server_history.cpp 1 63.7%
test/test_lang_bind_helper.cpp 2 93.2%
test/test_sync.cpp 2 92.99%
src/realm/util/future.hpp 3 95.94%
test/sync_fixtures.hpp 3 75.56%
src/realm/util/file.cpp 4 84.84%
src/realm/sync/instruction_applier.cpp 7 68.01%
src/realm/index_string.cpp 8 84.63%
src/realm/sync/noinst/server/server.cpp 8 73.85%
Totals Coverage Status
Change from base Build 2429: 0.004%
Covered Lines: 214771
Relevant Lines: 236083

💛 - Coveralls

@tgoyne tgoyne merged commit c1bb3a5 into master Jun 18, 2024
39 checks passed
@tgoyne tgoyne deleted the tg/download-progress branch June 18, 2024 20:08
@github-actions github-actions bot mentioned this pull request Jun 28, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 18, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Store FLX download progress estimate in the Realm file
2 participants