J S
37cd34f0fd
Merge pull request #2289 from sl-at-ibm/SL-data-api-adapter-improvements
...
Major Data API adapter overhaul (milestone 1)
2026-09-28 15:38:08 -05:00
Stefano Lottini
339aa7b94d
last touches to the self-documenting all-ops yamls
2026-09-28 10:30:22 +02:00
Stefano Lottini
401489148a
correct workload name in workload instructions
2026-09-28 10:30:22 +02:00
Stefano Lottini
3cb8ae0694
remove extensive data api logging for PR
2026-09-28 10:30:22 +02:00
Stefano Lottini
92009524ef
dataapi adapter documentation
2026-09-28 10:30:22 +02:00
Stefano Lottini
8a3b2d60b1
split legacyGetSortFromOp/getSortFromOp and reinstate legacy sort for legacy ops
2026-09-28 10:30:22 +02:00
Stefano Lottini
aca86cdd7d
add vectorize support to all sort clauses
2026-09-28 10:30:22 +02:00
Stefano Lottini
e6adad0016
uniform handling of vectors: list/comma-string/bindings, always bin-encoded in payloads - for all commands
2026-09-28 10:30:22 +02:00
Stefano Lottini
868bb4630d
data api: new findAndRerank op
2026-09-28 10:30:22 +02:00
Stefano Lottini
710f0e6a85
delete_collection, new and legacy
2026-09-28 10:30:22 +02:00
Stefano Lottini
b99c31a397
reinstated the new/legacy createCollection ops after a revert-merge-commit git mess
2026-09-28 10:30:22 +02:00
Stefano Lottini
f381e22beb
legacy createCollection: allow vecze params without apikey
2026-09-28 10:30:22 +02:00
Stefano Lottini
4945fe5480
insertMany, legacy+newstyle reworked entirely
2026-09-28 10:30:22 +02:00
Stefano Lottini
68d90231c7
reworked insertOne and 'insert one vector' (legacy+newstyle)
2026-09-28 10:30:22 +02:00
Stefano Lottini
7117dfaf63
updateMany reworked (legacy+newstyle)
2026-09-28 10:30:22 +02:00
Stefano Lottini
61c517e21a
replaceOne reworked (only legacy, it's not a data api primitive)
2026-09-28 10:30:22 +02:00
Stefano Lottini
917524490a
create and use getReplacementFromOp
2026-09-28 10:30:22 +02:00
Stefano Lottini
a2a4be806d
findOneAndReplace, fully reworked
2026-09-28 10:30:22 +02:00
Stefano Lottini
73ad2a44f4
find_one_and_delete: legacy/new split, revise syntax parsing - done
2026-09-28 10:30:22 +02:00
Stefano Lottini
abbd60060b
new-style updateOne added
2026-09-28 10:30:22 +02:00
Stefano Lottini
cfbf9393ee
update1 fully made legacy
2026-09-28 10:30:22 +02:00
Stefano Lottini
5b438f5356
countDocuments, add new op and split from legacy
2026-09-28 10:30:21 +02:00
Stefano Lottini
459ab94d4a
estimatedDocumentCount (no in-dispenser changes needed)
2026-09-28 10:30:21 +02:00
Stefano Lottini
7019fe9e7e
deleteOne and deleteMany: moved old ops to legacy, created new ops, added deprecation - completed
2026-09-28 10:30:21 +02:00
Stefano Lottini
e0a0458d46
fix wrong allow/deny-indexing representation in create_collection_with_class
2026-09-28 10:30:21 +02:00
Stefano Lottini
66f628905c
deprecate legacy finds; introduce new find to replace them
2026-09-28 10:30:21 +02:00
Stefano Lottini
3c317a4dbb
legacy find-based ops, confining to legacy state
2026-09-28 10:30:21 +02:00
Stefano Lottini
d2633b3dea
legacy find-based ops, confining to legacy state WIP
2026-09-28 10:30:21 +02:00
Stefano Lottini
35a8460490
just remove commented code
2026-09-28 10:30:21 +02:00
Stefano Lottini
8722ca69bb
add new-style find_one (only op dispenser differs. freeform filter)
2026-09-28 10:30:21 +02:00
Stefano Lottini
8a5d394570
find_one: fix projection, add inclSim, reinstate filters
2026-09-28 10:30:21 +02:00
Stefano Lottini
615b3ff3c9
sort op mapper and optype enum, fix filter in legacy f1upd, fix find_distinct by splitting into getLegacyFilterFromOp
2026-09-28 10:30:21 +02:00
Stefano Lottini
253a47dd7b
introduce deprecation warning for explicitly-Legacy Ops
2026-09-28 10:30:21 +02:00
Stefano Lottini
69313d2deb
switched to free-form in filter+update for (new) find_one_and_update op
2026-09-28 10:30:21 +02:00
Stefano Lottini
abd8322b51
restored all legacy ops from main; full split f1upd/legacy f1upd
2026-09-28 10:30:21 +02:00
Stefano Lottini
9a634d7da9
introduced getUpsertFromOp; enable upsert and sort in find_one_and_update
2026-09-28 10:30:21 +02:00
Stefano Lottini
fcd8d9a35e
expand the find_one_and_update syntax: legacy 'updates:update:...' with deprecation + new reasonable syntax
2026-09-28 10:30:21 +02:00
Stefano Lottini
4128a7c41d
find_one_and_update accepts non-doubles for inc and min
2026-09-28 10:30:21 +02:00
Stefano Lottini
0faa7d7e46
remove create_collection_with_class and find_distinct out-of-scope data api operations
2026-09-28 10:30:21 +02:00
Stefano Lottini
e5cafec8d9
align DataAPIOpType enum and the corresponding op mapper case clauses
2026-09-28 10:30:21 +02:00
Stefano Lottini
6621625c0d
TEMPorarily instrument Data API client to debug-log requests/responses
2026-09-28 10:30:21 +02:00
Stefano Lottini
8768ea7843
getSortFromOp always validates against the ascdesc+vector illegal double-sort and returns at most one
2026-09-28 10:30:21 +02:00
Stefano Lottini
334df15d69
add package-info.java for data api adapter
2026-09-28 10:30:21 +02:00
Stefano Lottini
c054d59075
asc/desc sort and vector sort are handled by one and the same getSortFromOp shared utility
2026-09-28 10:30:21 +02:00
Stefano Lottini
f68d0b2be3
bump data api client to 2.3.0
2026-09-28 10:30:21 +02:00
Stefano Lottini
dbfb1db00f
build-time check that no AND and OR are mixed in the data api filters
2026-09-28 10:30:21 +02:00
Stefano Lottini
c672b2bafe
Data API adapter: numeric filter operators accept integer values less than 2^32
2026-09-28 10:30:21 +02:00
Jonathan Shook
9dbbb38c7a
test(virtdata): state format_version 2 on the sharded bindings fixture
...
An unversioned manifest is held to version 1 and refused when it carries a multi-file facet, the rule the loader took from the reference this morning. The fixture predates that rule and now states the version its content needs, as every sharded fixture in nb-vectordata does.
2026-09-16 08:24:57 +00:00
Jonathan Shook
56f342d9a1
bump version
2026-09-16 07:17:44 +00:00
Jonathan Shook
5e2b6dd2e3
docs(vectordata): explain the recall regime and low-cardinality degradation in the remote access guide
...
A filtered ground truth is only as deep as each predicate's match set, so a selectivity ladder cut for a full dataset walks off the bottom of a small slice and pads its rows with sentinels; the strict recall then caps at the profile's ceiling however well the search does. The guide derives that ceiling, tabulates the three measured runs, and states the regime: survey the ground truth before loading, report the attainable measure beside the strict one, and choose the profile by what the survey says it can pay out.
2026-09-16 07:16:51 +00:00
J S
Stefano Lottini