Commit Graph

973 Commits

Author SHA1 Message Date
Christien Rioux
b46fd7690f watch maintenance tasks 2024-01-19 13:53:42 -05:00
Christien Rioux
a67bfde1f7 send valuechanged 2024-01-19 13:53:42 -05:00
Christien Rioux
d1dad8de61 checkpoint 2024-01-19 13:53:42 -05:00
Christien Rioux
9b8420d288 more watchvalue 2024-01-19 13:53:42 -05:00
Christien Rioux
8efa2de82c config -> record store 2024-01-19 13:53:42 -05:00
Christien Rioux
077a1808a5 propagate set logic to watch logic 2024-01-19 13:53:42 -05:00
Christien Rioux
4a76353f33 ensure get_value doesn't try to pull values unless they are close enough 2024-01-19 13:53:42 -05:00
Christien Rioux
a211c7cce3 more watchvalue 2024-01-19 13:53:42 -05:00
Christien Rioux
248f8dad06 watchvalue recordkeeping 2024-01-19 13:53:42 -05:00
Christien Rioux
9d9a76e45c more watch value 2024-01-19 13:53:42 -05:00
Christien Rioux
70e256a25a checkpoint 2024-01-19 13:53:42 -05:00
Christien Rioux
3f86801ecd Merge branch 'salvatoret/clean-up-warnings' into 'main'
Clean up the compile-time warnings

See merge request veilid/veilid!247
2024-01-19 18:53:31 +00:00
Salvatore Testa
5884f89d1a
Clean up the compile-time warnings
These are all the auto-applied warning corrections.
2024-01-11 15:16:18 -08:00
Salvatore Testa
e378d01682
Move default storage config to veilid-core
The default is currently `""` which puts all of the files without
grouping them in the top level.

Instead, use the paths that `veilid-server` has configured as the
defaults.
2024-01-11 10:19:31 -08:00
Christien Rioux
ead8e9ab86 Merge branch 'salvatoret/more-defaults' into 'main'
Populate config defaults with more helpful values

See merge request veilid/veilid!244
2024-01-10 00:31:56 +00:00
Salvatore Testa
37530b9128
Populate config defaults with more helpful values
Make it easier for someone to spin up a Veilid node.

I copied a lot of these from the `default_config.dart`.
2024-01-08 17:44:41 -08:00
Salvatore Testa
7a55be8015
Add VeilidConfigInner-based VeilidAPI startup
Allow for a type-safe initialization of the API

Closes https://gitlab.com/veilid/veilid/-/issues/354
2024-01-06 13:21:44 -08:00
Christien Rioux
bd74ac7bce Merge branch 'salvatoret/android-dev-2023-11' into 'main'
Update the Android development enviornment setup

See merge request veilid/veilid!237
2023-11-23 14:51:38 +00:00
Kyle H
af27b5aa85 Change 'whitelist' to 'allowlist' globally. 2023-11-23 14:49:45 +00:00
Salvatore Testa
38d9610c6b
Update the Android development enviornment setup
1. Update `ANDROID_SDK_ROOT` to `ANDROID_HOME`. The former variable is
   deprecated. (see https://developer.android.com/tools/variables#envar)
2. Remove `ANDROID_NDK_HOME` environment variable. This should allow the
   build script to work out of the box for more folks.
3. Check that Java is on the `PATH` as opposed to just installing it.
   `asdf` and other runtime management tools are pretty popular, and all
   we care about is that the Java version is accessible.
4. Remove calls to `sudo`. Check to see if CocoaPods exists, if it
   doesn't install it using Homebrew which doesn't require `sudo`.
2023-11-19 17:31:23 -08:00
Christien Rioux
6f8f636d5d disable dht test in CI 2023-11-08 08:04:53 -05:00
Christien Rioux
c67a427610 mtime fixes 2023-11-07 21:06:29 -05:00
Christien Rioux
ffacbdfe25 clarify error 2023-11-07 20:38:36 -05:00
Christien Rioux
425b31c023 make capnp only necessary if modifying generated files 2023-11-07 19:27:08 -05:00
Christien Rioux
1e4e9c203c check in generated files 2023-11-07 14:19:28 -05:00
Christien Rioux
5070b0a560 use forked repositories for async-tls and async-tungstenite 2023-11-07 13:49:19 -05:00
Christien Rioux
8515e0dccc cargo update 2023-11-06 22:49:20 -05:00
TC Johnson
dcd4c0d22a
Version update: v0.2.4 -> v0.2.5
ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
2023-11-06 20:51:41 -06:00
Christien Rioux
2dee3ff6cc allow returning local values when offline, even when forcerefresh is turned on 2023-11-05 21:16:13 -05:00
Christien Rioux
765ecee450 Merge branch 'safety-by-default' into 'main'
Safety by default

See merge request veilid/veilid!235
2023-11-06 01:40:02 +00:00
Christien Rioux
071d294a70 use Reliable and PreferOrdered by default for private routes 2023-11-05 19:48:33 -05:00
Christien Rioux
ee375ad430 safety by default 2023-11-05 18:38:05 -05:00
Christien Rioux
1b380cd915 Merge branch 'kyanha-safety-route-bidi' into 'main'
Make private route allocation bidirectional

Closes #341

See merge request veilid/veilid!231
2023-11-05 14:30:10 +00:00
Christien Rioux
ee1472f3ae fix #337 2023-11-04 19:53:14 -04:00
Christien Rioux
38cd29f9a1 wasm refactor for flows 2023-11-04 19:53:14 -04:00
Christien Rioux
0640342556 refactor for flows 2023-11-04 19:53:14 -04:00
Christien Rioux
facb343160 clean up protect/refs 2023-11-04 19:53:14 -04:00
Christien Rioux
934cd93b4b cleanup 2023-11-04 19:53:14 -04:00
John Smith
7f67a00ab2 clean up close ordering 2023-11-04 19:53:14 -04:00
Christien Rioux
520d8c43f7 connection debugging 2023-11-04 19:53:14 -04:00
Christien Rioux
f47d6402c3 staggered pings 2023-11-04 19:53:14 -04:00
Christien Rioux
d750b7c5c3 wasm dht test passes 2023-11-04 19:53:14 -04:00
Christien Rioux
11c19d1bad wasm warnings 2023-11-04 19:53:14 -04:00
John Smith
b590e3f28a signal work" 2023-11-04 19:53:14 -04:00
Christien Rioux
e677a72ab3 oops Please enter the commit message for your changes. Lines starting 2023-11-04 19:53:14 -04:00
Christien Rioux
74e2f9a2c0 cleanup visibility 2023-11-04 19:53:14 -04:00
Christien Rioux
2051292a26 move file 2023-11-04 19:53:14 -04:00
Christien Rioux
b964ddb6eb protect route hops + refactor 2023-11-04 19:53:14 -04:00
John Smith
c70c260bb8 add low level close 2023-11-04 19:53:14 -04:00
Christien Rioux
23f47be357 turn off linger 2023-11-04 19:53:14 -04:00
Christien Rioux
2c4883ee85 fix tests 2023-11-04 19:53:14 -04:00
Christien Rioux
126bb0035d refactor wasm tests 2023-11-04 19:53:14 -04:00
Christien Rioux
8fa4ab10a4 work on wasm tests 2023-11-04 19:53:14 -04:00
TC
502d08eda3 Merge branch 'earthly-caching-2' into 'main'
Refactored Earthfile to improve caching

See merge request veilid/veilid!232
2023-11-03 01:17:14 +00:00
Δ ǀ Ξ ȼ
d38f3cb42f
Refactored Earthfile to improve caching
Targets have been parallelized so that initial push of the container cache should build the whole build a little quicker, plus the container should now use the cache for more of the build and so speed up normal builds to just the compilation and test of the code that has changed

The `build_cache` target now builds a `build-cache:latest` container that is stored in the GitLab project Container Registry, and then used (if it exists) by the `test_build` target.  The `build_cache` task runs under 3 conditions, 1. the container does not exist, 2. if scheduled, 3. if run manually from the Pipelines page in the GitLab interface.

It is recommended that the build is set up to run on a weekly schedule via the `Pipeline schedules` page in GitLab with the schedule of `0 2 * * 6`.
2023-10-31 11:17:13 +00:00
Kyle H
6ee78700f1 Make private route allocation bidirectional 2023-10-28 04:47:31 -04:00
Christien Rioux
5691e5a6ac Merge branch 'rust-dht-tests' into 'main'
Rust DHT test suite

See merge request veilid/veilid!227
2023-10-21 23:29:09 +00:00
Sergei Surovsev
992cc7a1fd Rust DHT test suite 2023-10-21 23:29:09 +00:00
Christien Rioux
6b4bcc67f4 fixes 2023-10-21 19:22:59 -04:00
Christien Rioux
7da6b62c52 more cleanup 2023-10-21 18:51:16 -04:00
Christien Rioux
48ab34f577 clean up error, make python tests more reliable 2023-10-21 15:54:07 -04:00
Christien Rioux
697ac5e9ce error cleanup 2023-10-21 15:00:50 -04:00
Christien Rioux
97be49a9a7 clean up handling of errors in route spec store 2023-10-20 22:39:09 -04:00
Christien Rioux
353c907497 more log 2023-10-19 23:09:42 -04:00
Christien Rioux
2a1b3762c8 dont blow away routes every 5 seconds 2023-10-19 23:09:42 -04:00
Christien Rioux
c4b8985263 support WSS IP Urls 2023-10-18 14:50:48 -04:00
Christien Rioux
5757a5badf rustls update 2023-10-18 14:50:42 -04:00
Christien Rioux
edd2ec20ba one more case 2023-10-18 07:40:16 -04:00
Christien Rioux
6f23c4bd0e fix doc build for docs.rs 2023-10-17 22:13:00 -04:00
Christien Rioux
5468e5ce9d capnp warning the right way 2023-10-16 15:10:05 -04:00
Christien Rioux
f8bd8c6e65 protoc warning the right way 2023-10-16 15:09:00 -04:00
TC Johnson
cc5cb8a8b0
Version update: v0.2.3 → v0.2.4 2023-10-15 12:24:59 -05:00
John Smith
855b7eaf7d ipv6 detection 2023-10-15 11:27:31 -04:00
Christien Rioux
a363bb0462 fix wasm32 unit tests 2023-10-14 15:59:12 -04:00
Christien Rioux
8102db1d06 Merge branch 'opeationTypo2023-01-14' into 'main'
Fix opeation in operation_waiter.rs comment

See merge request veilid/veilid!224
2023-10-14 18:13:43 +00:00
Christien Rioux
a03c84cff8 Merge branch 'capabiltiiesTypo2023-10-14' into 'main'
Fix capabiltiies comment typo find_peers.rs

See merge request veilid/veilid!223
2023-10-14 18:13:33 +00:00
Christien Rioux
4de532c797 Merge branch '1aferTypo2023-10-14' into 'main'
Fix "bytes afer" comment typo byte_array_types.rs

See merge request veilid/veilid!221
2023-10-14 18:13:07 +00:00
Christien Rioux
794c6306a6 Merge branch 'boostrapTypo2023-10-14' into 'main'
Fix boostrap in bootstrap.rs comment

See merge request veilid/veilid!220
2023-10-14 18:12:56 +00:00
Christien Rioux
2fc4247be6 try different dialinfo for bootstrap 2023-10-14 13:50:34 -04:00
Haley Weslin
220ed57fdd Fix opeation in operation_waiter.rs comment
This fixes a typo of "opeation" for "operation" in a comment
in veilid-core/src/rpc_processor/operation_waiter.rs.
2023-10-14 17:39:12 +00:00
Haley Weslin
2ea16c4aa8 Fix capabiltiies comment typo find_peers.rs
This fixes a typo of capabiltiies for capabilities in
veilid-core/src/routing_table/find_peers.rs.
2023-10-14 17:33:18 +00:00
Haley Weslin
c6d434ca8d Fix "bytes afer" comment typo byte_array_types.rs
This fixes a typo of "bytes afer" for "bytes after" in a comment
in veilid-core/src/crypto/types/byte_array_types.rs.
2023-10-14 17:20:40 +00:00
Haley Weslin
514f9aa60d Fix boostrap in bootstrap.rs comment
This fixes a typo of "boostrap" for "bootstrap" in a comment
in veilid-core/src/routing_table/tasks/bootstrap.rs.
2023-10-14 17:13:44 +00:00
John Smith
cf6a139928 linux fixes 2023-10-13 18:17:10 -04:00
John Smith
d922bc1f5d windows upnp work 2023-10-13 17:57:38 -04:00
Christien Rioux
ebd36d82ef network interface changes cleanup 2023-10-12 21:17:47 -04:00
Christien Rioux
54c403ebfb keep network from going down when external ip addresses haven't changed 2023-10-11 23:12:54 -04:00
Christien Rioux
084d4ae98b remove crap 2023-10-10 09:56:56 -04:00
Christien Rioux
c80a473fdb missing symbols on x86_64 android emulator 2023-10-10 09:56:56 -04:00
Christien Rioux
86dee2b6ed xfer 2023-10-10 09:56:56 -04:00
Trent Waddington
72889b8f1a Fix warnings. 2023-10-10 12:51:11 +10:00
John Smith
1fec1a5008 windows cleanup 2023-10-07 23:00:28 -04:00
Bruno Bigras
20442f7b9a fix typos 2023-10-05 12:11:42 -04:00
Christien Rioux
f59c4509ea rework public address detection timing 2023-09-30 22:42:06 -04:00
Christien Rioux
3d50f7ef11 Merge branch 'typo' into 'main'
fix minor typo

See merge request veilid/veilid!209
2023-09-29 19:56:02 +00:00
Christien Rioux
0120b99ae0 Merge branch 'onceTypo2023-09-28' into 'main'
Fix "have been the once" in bucket_entry.rs

See merge request veilid/veilid!207
2023-09-29 19:55:05 +00:00
Christien Rioux
0528eda9ff Merge branch 'evenTypo2023-09-28' into 'main'
Fix "in the even we" in public_address_check.rs

See merge request veilid/veilid!208
2023-09-29 19:54:26 +00:00
Christien Rioux
27c0f6e73a fix dial info filter 2023-09-29 09:38:12 -04:00
Derrick Oswald
5f3fb954d8 fix minor typo
Fixes the Progressive Web App configuration doc comment which has a typo in the word Progressive.
2023-09-29 08:11:49 +02:00
Christien Rioux
91031531e4 address type detection 2023-09-28 21:54:31 -04:00
Haley Weslin
b8980756b2 Fix "in the even we" in public_address_check.rs
This fixes a wordo (even for event) in public_address_check.rs.
2023-09-29 01:32:45 +00:00
Haley Weslin
6883607ff8 Fix "have been the once" in bucket_entry.rs
This fixes a wordo (once for one) in bucket_entry.rs.
2023-09-29 00:59:36 +00:00
Christien Rioux
bbfd21edee some debugging 2023-09-28 15:44:06 -04:00
Christien Rioux
b9505c031c xfer 2023-09-28 12:52:49 -04:00
Christien Rioux
069926aba8 fixes 2023-09-25 22:59:41 -04:00
Christien Rioux
2d075626f1 fixes 2023-09-25 22:59:41 -04:00
John Smith
ca1a802b5b remove spurious error from logs 2023-09-25 22:59:41 -04:00
Christien Rioux
bd4b4233bf Merge branch 'itsTypo2023-09-25' into 'main'
Fix non-contraction it's in set_value.rs

See merge request veilid/veilid!204
2023-09-26 02:51:36 +00:00
Christien Rioux
d817d287f0 Merge branch 'possibleTypo2023-09-25' into 'main'
Fix "can possible be" in rpc_processor/mod.rs

See merge request veilid/veilid!203
2023-09-26 02:51:23 +00:00
Christien Rioux
3e8f2d315b Merge branch 'searingTypo2023-09-25' into 'main'
Fix "searing" in fanout_call.rs comment

See merge request veilid/veilid!202
2023-09-26 02:51:10 +00:00
Christien Rioux
c278330c67 Merge branch 'nodeTypo2023-09-21' into 'main'
Fix "node will node" in fanout_call.rs comment

See merge request veilid/veilid!199
2023-09-26 02:50:54 +00:00
Haley Weslin
9d60ac01d0 Fix non-contraction it's in set_value.rs
This uses "its" in place of "it's" when not a contraction.
2023-09-25 19:20:57 +00:00
Haley Weslin
073622b8bf Fix "can possible be" in rpc_processor/mod.rs
This fixes a wordo of possible for possibly.
2023-09-25 19:13:16 +00:00
Haley Weslin
531cb71d00 Fix "searing" in fanout_call.rs comment
This fixes a wordo of searing for searching.
2023-09-25 18:59:30 +00:00
Christien Rioux
38ff6f81c5 Merge branch 'feature/bytes-as-uint8arrays' into 'main'
(wasm) Treat arbitrary byte data as Uint8Array, instead of base64url marshalling.

See merge request veilid/veilid!196
2023-09-23 16:50:27 +00:00
Haley Weslin
629f521c47 Fix "node will node" in fanout_call.rs comment
This fixes "node" wordo in a comment: the logic of the software implies
"node will not" here.
2023-09-21 22:28:49 +00:00
Christien Rioux
0a7f21ce00 update keyvaluedb 2023-09-21 15:53:25 -04:00
Christien Rioux
2627a22597 32 bit fix 2023-09-20 11:31:01 -04:00
Brandon Vandegrift
80afa19678 (wasm) Treat arbitrary byte data as Uint8Array, instead of base64url marshalling. 2023-09-20 00:46:45 -04:00
Christien Rioux
0bf595c53a simplify version checking 2023-09-19 19:12:51 -04:00
Christien Rioux
563b80e31c Merge branch 'clippy-fixes' into 'main'
Clippy fixes

See merge request veilid/veilid!195
2023-09-19 00:43:41 +00:00
Christien Rioux
20451af880 more clippy 2023-09-18 19:49:57 -04:00
Christien Rioux
f596b3ce05 more clippy 2023-09-18 15:22:40 -04:00
Christien Rioux
c7d4462e0e more clippy 2023-09-17 22:12:25 -04:00
Christien Rioux
e4ee093951 clippy work 2023-09-17 20:45:30 -04:00
Christien Rioux
6438a64fc7 clippy work 2023-09-17 19:37:02 -04:00
TC Johnson
fa13f95a10
Version update: v0.2.2 → v0.2.3
- Security fix for WS denial of service
- Support for latest Rust 1.72
2023-09-17 13:21:57 -05:00
Christien Rioux
8a1260ed48 clippy --fix 2023-09-17 14:14:39 -04:00
Christien Rioux
1f5867890d fix WS security issue 2023-09-17 13:32:02 -04:00
John Smith
d60c296303 cargo upgrades 2023-09-16 21:56:19 -04:00
John Smith
e768e79961 cargo workspace resolver update for latest rust 2023-09-16 20:28:43 -04:00
TC Johnson
06c0abead8
Version update: v0.2.1 → v0.2.2 2023-09-16 17:32:16 -05:00
Christien Rioux
eda99bbd47 node search fix 2023-09-16 16:51:59 -04:00
Christien Rioux
960a4cb964 Revert "version bump fix"
This reverts commit 2d291b86b6.
2023-09-16 16:51:59 -04:00
Christien Rioux
ba72aa9dd3 version bump fix 2023-09-16 16:51:59 -04:00
Christien Rioux
7f3f7c9166 clean up cargo.toml 2023-09-16 16:51:59 -04:00
John Smith
177bccbe12 windows build fix 2023-09-16 16:51:58 -04:00
Christien Rioux
ad47e400ed build fixes 2023-09-16 16:51:58 -04:00
Christien Rioux
3a4c211e85 resolve work 2023-09-16 16:51:58 -04:00
John Smith
dad8e50939 oops 2023-09-16 16:51:58 -04:00
Christien Rioux
c01be8f62d capnp and protoc versioning 2023-09-16 16:51:57 -04:00
Christien Rioux
17ea68fb84 connection fix 2023-09-14 15:51:09 -04:00
Christien Rioux
bce74026d3 Merge branch 'more-debug-commands' into 'main'
Debugging

See merge request veilid/veilid!188
2023-09-14 18:06:34 +00:00
Christien Rioux
86aae741e0 reverse connect for wasm nodes 2023-09-13 17:00:39 -04:00
Christien Rioux
671f349578 conn table 2023-09-12 21:40:13 -04:00
Brandon Vandegrift
c2c607efac (wasm) Better TS types for serialized structs, RoutingContext++, more crypto fns 2023-09-11 18:42:05 -04:00
Christien Rioux
5b3d589bf2 print max 2023-09-10 20:36:39 -04:00
Christien Rioux
b3294d905d rename 2023-09-10 20:30:29 -04:00
Christien Rioux
6e26fb4c8f more nodeinfo 2023-09-10 20:16:57 -04:00
Christien Rioux
a5e17a0d65 appreply 2023-09-10 17:07:22 -04:00
Christien Rioux
3fdf12e536
Merge branch 'target_env_musl' into 'main'
Adding musl target env

See merge request veilid/veilid!177
2023-09-10 21:04:00 +00:00
Christien Rioux
28b08034f5 appmessage/call commands 2023-09-10 15:53:19 -04:00
Christien Rioux
b51e14783b resolve node 2023-09-10 11:56:50 -04:00
Christien Rioux
607b595807 fixes 2023-09-10 11:24:52 -04:00
Christien Rioux
2ff2ab7aa8 debugging for public address 2023-09-09 22:34:42 -04:00
Christien Rioux
60a7e90712 dht fixes 2023-09-09 21:44:16 -04:00
Christien Rioux
07f92b6e3f more setvalue fixes and concurrency cleanup 2023-09-09 18:35:25 -04:00
Christien Rioux
853976789f fix crash and clean up record data size housekeeping 2023-09-09 13:30:48 -04:00
Christien Rioux
36957d84f1 fix bug 2023-09-08 20:50:05 -04:00
Christien Rioux
0aa7cf5ef2 fanout work 2023-09-08 20:38:31 -04:00
Linus Gasser
0ac6fc7dc7 Adding musl target env 2023-09-08 19:23:46 +02:00
Christien Rioux
dc17e25bac Fixes #297
WASM build for new public address detection code
2023-09-06 17:06:33 -04:00
Christien Rioux
bfc42cdd8e clean up ui
pause routing table tasks when making routing domain changes
2023-09-06 13:20:36 -04:00
Christien Rioux
b325c82b9a more parallel 2023-09-06 10:02:55 -04:00
Christien Rioux
dfcdcf2364 xfer 2023-09-06 10:02:55 -04:00
Christien Rioux
d5a4be8a36 missing file 2023-09-06 10:02:55 -04:00
Christien Rioux
dac8e75229 xfer 2023-09-06 10:02:55 -04:00
Christien Rioux
3dfb612e6f fix ws 2023-09-06 10:02:55 -04:00
Christien Rioux
80b2e7b9da pad work 2023-09-06 10:02:55 -04:00
Christien Rioux
1b5934dad4 public address detection refactoring 2023-09-06 10:02:55 -04:00
John Smith
8d694f20cd fix async std test and make windows test script work 2023-09-06 09:53:28 -04:00
Christien Rioux
4b298f43d4
Merge branch 'settingTypo2023-09-05' into 'main'
setting not settings: struct.RoutingContext.html

See merge request veilid/veilid!173
2023-09-06 12:59:45 +00:00
Haley Weslin
58524657fe setting not settings: struct.RoutingContext.html
"by settings ‘force_refresh’" needs to use the "setting" word.

After changing it, build_docs.sh fixes rustdoc for
veilid_core/struct.RoutingContext.html
2023-09-05 16:05:18 +00:00
Brandon Vandegrift
93963e9d08 (wasm) Cleanup and refactoring, add TableDB transactions, copy over docs for JSDoc. 2023-09-04 14:04:57 -04:00
Brandon Vandegrift
ca11f6075d (wasm) Add VeilidCrypto class, refine interfaces for VeilidRoutingContext 2023-09-03 23:27:20 -04:00
Brandon Vandegrift
c5d7922fc5 Add VeilidRoutingContext class for WASM 2023-09-03 23:27:20 -04:00
Christien Rioux
9aeec3cfa8 build fixes and cleanup 2023-09-03 23:27:20 -04:00
Brandon Vandegrift
779532b624 Add new VeilidTable and VeilidClient to WASM API
These new interfaces have been reworked (compared to the original API) to emit (mostly) proper TypeScript types.
2023-09-03 23:27:08 -04:00
Brandon Vandegrift
1b20037053 Setup for TypeScript type gen for WASM using Tsify
- Includes breaking changes to the WASM API surface, since it now accepts objects instead of stringified JSON.
2023-09-03 23:27:08 -04:00
TC Johnson
65826b219b
Version update: v0.2.0 → v0.2.1 2023-09-03 19:45:12 -05:00
John Smith
71f91998b2 more fork 2023-09-03 10:23:41 -04:00
John Smith
896df0bb97 more forking 2023-09-03 10:08:04 -04:00
John Smith
fd6e9e22c2 crate fixes 2023-09-03 09:53:00 -04:00
Christien Rioux
7d71b50170
Merge branch 'rustdocTypo2023-09-02' into 'main'
Remove stray by: veilid_core/struct.VeilidAPI.html

See merge request veilid/veilid!166
2023-09-03 00:18:16 +00:00
Christien Rioux
b3354194e0 repo fix 2023-09-02 18:50:12 -04:00
Christien Rioux
a77f80a8a9 repo fixes 2023-09-02 15:47:19 -04:00
Christien Rioux
20f55745b0 add descriptions 2023-09-02 12:28:31 -04:00
TC Johnson
2c46a159a3
Version update: v0.1.10 → v0.2.0 2023-09-02 09:45:56 -05:00
Christien Rioux
90772728c2 bumpversion fixes 2023-09-02 10:41:59 -04:00
Haley Weslin
3cfaff9cbb Remove stray by: veilid_core/struct.VeilidAPI.html
"specified by at startup time" has an extra "by" (wrong grammar)

After changing it, build_docs.sh fixes rustdoc here:
target/doc/veilid_core/struct.VeilidAPI.html
2023-09-02 05:22:51 +00:00
Christien Rioux
709ec4543b change visibility of dhcache 2023-09-01 21:18:45 -04:00
Christien Rioux
246056913e doc work 2023-09-01 21:13:05 -04:00
Christien Rioux
c377a59278 fix tests 2023-09-01 18:59:43 -04:00
Christien Rioux
ffcf16711b android fixes 2023-09-01 17:44:42 -04:00
Christien Rioux
d68a543800 fix ios 2023-08-29 17:50:25 -05:00
Christien Rioux
e302b764d0 docs and tests work 2023-08-29 15:15:47 -05:00
Christien Rioux
3125c19f02 doc work 2023-08-27 16:39:50 -05:00
Christien Rioux
8c366387eb Merge branch 'fix-set-record-data-size-call' into 'main'
call set_record_data_size with accumulated size

See merge request veilid/veilid!155
2023-08-27 16:07:38 +00:00
Christien Rioux
59dda0febe Merge branch 'api_startup_json_skip_all' into 'main'
api_startup_json: use 'skip_all'

See merge request veilid/veilid!154
2023-08-27 15:58:00 +00:00
Rivka Segan
5dd0a3793b call set_record_data_size with accumulated size
set_subkey corrupts record_data_size in a Record struct by calling
set_record_data_size with a value that depends only on the length of
the new subkey value. This leads to various undesirable outcomes, such
as: applications can write more than MAX_RECORD_DATA_SIZE without
encountering the intended "veilid.error.VeilidAPIErrorGeneric:
label='Generic' message='dht record too large'" error message, and
"panicked at 'attempt to subtract with overflow'" (i.e., an attempt to
set a negative value of a size) if a subkey's new length is less than
a subkey's old length. Typically, record_data_size in a Record struct
will be incorrect if a value was set for more than one subkey. Some
users might want to start over with a table_store that doesn't have
any incorrect record_data_size values.

The issue begins here:
6f71c6a00a/veilid-core/src/storage_manager/record_store.rs (L583-L586)

and is triggered here:
6f71c6a00a/veilid-core/src/storage_manager/record_store.rs (L613-L615)

It should be clear that new_record_data_size is only related to the
subkey that is currently being set. The amount of data in the record,
before set_subkey is called, is ignored. It appears that
new_total_size, not new_record_data_size, was intended to be used for
set_record_data_size, and this change succeeds for me in limited
testing but I don't have a comprehensive test suite.

One way to reproduce is by running the code from
https://gitlab.com/vatueil/veilid-file on a greater than 1 MB file
while watching variable values within
veilid-core/src/storage_manager/record_store.rs. For example: "poetry
run file put /usr/bin/tcpdump" (1.3 MB on Ubuntu 23.04). With the
original Veilid code, each of the dozens of subkey writes is checking
whether a roughly 32K value is greater than 1048576, it never is, and
thus there is never a "dht record too large" error. With the patch in
this MR, each of the dozens of subkey writes is checking whether an
ever-increasing value is greater than 1048576, it eventually is, and
the "dht record too large" error is printed. With the patch, one can
work with smaller files, e.g., do "poetry run file put /usr/bin/ssh"
(0.8 MB) followed by "poetry run file get VLD0:<_insert_key_here_>
ssh-copy" and the retrieved file ssh-copy is identical to
/usr/bin/ssh.

The more detailed behavior is that the modified code has
record.total_size of 350 on the first iteration, then 33596, 66842,
100088, etc. The original code also has record.total_size of 350 on
the first iteration, but then stays at 33246 forever (33246 is the
user-supplied subkey size of 32768, plus 128, plus the minimum record
size of 350),
2023-08-26 07:08:47 +00:00