* Support non exception errors from fetcher
* revert parital changes to store builder to reduce noise
* finish off diff
* Allow to create a FetcherResult.Error without a Throwable. Add tests
* Add missing funcion and more tests
* lint
* unflake RxFlowableStoreTest
* try to rename FakeFetcher to FakeRxFetcher to (maybe) solve missing codcov
* move SourceOfTruth out of impl package
* Rename accidental change of RxStoreBuilder.fromMaybe back to formSingle
* Introduce Fetcher from #139
* fix Rx artifact
* delete legacy presistor factory
* fix api file
* move fetcher to be a typealias
* code review comments + clean up documentation
* code review comments
* Update store/src/main/java/com/dropbox/android/external/store4/Fetcher.kt
Co-Authored-By: Yigit Boyar <yboyar@google.com>
* Revert "Update sample app's build.gradle to refer to the externally released version of Store (#159)"
This reverts commit fc8da86f63.
* update releasing.md
Co-authored-by: Yigit Boyar <yboyar@google.com>
* Contain @ExperimentalStdlibApi within relevant scope.
The alpha04 release of Store caused the requirement of the @ExperimentalStdlibApi annotation for client code. I looked into it, and the only thing in Store that required the annotation was the Kotlin ArrayDeque used in the BufferImpl class. That class is private in the ChannelManager.kt file, so I opted that file into the annotation and removed all other occurences of the annotation.
Code compiles, all 208 tests pass.
* update to correct git glitch in /build.gradle
* update to address 'needless blank line' linter
* Prepare for release 4.0.0-alpha05
* Prepare for next development version.
Co-authored-by: Dave Parker <davidp@dropbox.com>
* Migrate cache library to use kotlin.time APIs
* Fix docs and inline durations in tests
* Migrate cache library to use kotlin.time APIs
* Only require ExperimentalTime when setting cache expiration
* Set ExperimentalTime on RealStore to use it on MemoryPolicy
* Set ExperimentalTime on Cache interface and not on functions
* Avoid ExperimentalTime annotation in filesystem module
* Remove unneeded ExpirationTime annotation
* Remove unneeded ExpirationTime annotation
* Ensure network starts after disk is established
This PR fixes a bug in the ordering where if disk is skipped and network
is super fast, there is a tiny chance that we wouldn't latch onto source
of truth before triggering network and that would make SourceOfTruth think
that the value was written to disk before it observed, hence become a disk
value rather than fetcher.
This PR ensures that we unlock the network only after we establish a position
in the disk read queue so that we'll know the fetcher value came afterwards.
Fixes: 114
Test: RxSingleStoreTest
* Apply suggestions from code review
* move network unlock to disk flow setup for clarity
Co-authored-by: Mike Nakhimovich <digitalbuddha@users.noreply.github.com>
* more tests!
* Start tracking API binary compatibility (#97)
* Configure binary-compatibility-validator plugin.
* Generate current API dumps.
* Update CONTRUBUTON.md with instruction for updating API dumps.
* ktlint fix
* newlines
* put fs eraser back in
* more tests!
* ktlint fix
* newlines
* put fs eraser back in
* run api checks
* lint
* lint
Co-authored-by: Yang <ychescale9@gmail.com>
* [WIP] Add code coverage
* Get multi-module and merging working
* Add CodeCov
* Update location of report.xml
* Check instead of build depends on Jacoco
* Do not merge coverage
* Ignore tests
* update comment
* Fix typos.
* Gradle 6.1, AGP 4.0.0-alpha09, lifecycle 2.2.0.
* Add experimental invalidateAll() support and @ExperimentalStoreAPI annotation.
* Update sample with deleteAll function.
* Update README.md with deleteAll function.
* Add a section to README.md on clearing store entries.
* Rewrite tests without mocking. Move test utils / helpers to a single package.
* Code formatting and cleanups.
* Use StoreResponse.Data instead of DataWithOrigin in ClearAllStoreTest and ClearStoreByKeyTest.
* Simplified samples. Refactor tests.
* Gradle 6.1.1.
* Change flow collection util to drain the flow
This PR changes the assertItems util method in tests to drain the
flow to ensure we don't produce unwated items.
* reformat code
* Use a test subject for flow assertions
* update test subject messsages, add tests for it (so meta :p)
* Apply suggestions from code review
Co-Authored-By: Eyal Guthmann <eyalgu@users.noreply.github.com>
* fix code style
* Don't try to get exception unless collection finished
This fixes a bug in FlowSubject where it would try to get the cancelation
exception from an unfinished job. Also added a test.
Co-authored-by: Eyal Guthmann <eyalgu@users.noreply.github.com>
* Rewrite cache module in Kotlin.
* Migrate RealStore to use new cache.
* Migrate FileSystemImpl to use new cache.
* Migrate StoreTest to new cache.
* Remove Guava Cache.
* Migrate remaining Java file to Kotlin.
* Fix ktlint error.
* Fix package.
* Add support for CacheLoader.
* Remove unnecessary mockito-inline.
* Remove volatile keyword from CacheEntry fields and add docs re. cache entry reuse and thread-safty assumption.
* - Refactor CacheBuilder.
- Update docs.
- Update Key, Value, Input, Output generic types to be bound by Any.
- Replace spy with TestLoader for asserting invocation.
* Support key-based synchronization for get(key: Key, loader: () -> Value): Value.
* Document loader exception propagation.
* Small clean ups.
* Make CacheEntry's Key and Value bound to Any, move CacheEntry to RealCache.kt as a private class.
* Add reference counting key-based synchronization for cache loader.
* Improve KeyedSynchronizer and docs.
* Refactor tests and add a missing test case to reach 100% coverage.
* Add memory barrier after updating access time in cache entry.
* Only process access queue if expiresAfterAccess == true.
* Support configurable concurrency level.
* Increase delay for concurrency test.
* Explicitly set concurrency level for concurrency tests.
* Only add memory barrier after recordRead if expiresAfterAccess is enabled.
* Fix a potential memory leak caused by thread preemption.
* Allow retries (3 times) for concurrency tests.
* Move performMaintenance function to bottom of the class.
* Fix typos, rename performMaintenance() to cleanUpDeadEntries(), improve docs.
* Fix more typos.
* Only remove entries in map but not in queue.
* Replace memory barrier with @Volatile fields in CacheEntry.
* Fix deadlock while iterating sets.
* Remove stale cache/README.md, remove commented out java file.
* Add kotlin plugin to cache module
* Remove custom accessors in StoreDefaults as default values are effectively constant.
* Update JUnit to 4.13 RC2 to enable asserting specific error message on expected exception.
* Remove custom accessors in MemoryPolicy as all values are calculated from constants.
* Remove checkstyle and pmd as the codebase will be 100% Kotlin.
* Fix IDE warning in build.gradle by removing static imports.
* Remove unused guava dependencies.
* Improve StoreDefaults docs.
* Make MemoryPolicy Kotlin friendlier
* Make MemoryPolicy usage more Kotlin like
* Format code in FSReader
* Format code in FSWriter
* Format code in FileSystemRecordPersister
* Format code in FileSystemPersister
* Convert BarCodeReadAllPathResolver to Object instead of Class
* Convert BarCodePathResolver to Object instead of Class
* Fix wrong control flow conversion
* Clean up ChannelManager's interface to make it easier to add more tests (and add them)
Main changes:
* ~`ChannelManager` no longer inherits from `StoreRealActor` but rather
delegates to one.~ - Moved to followup PR
* messages coming from the upstream were placed under `Message.Dispach`
rather than `Message`. These are the only messages accepted from outside
`ChannelManager`
* `ChannelManager` now only exposes `plusAsign`, `minusAsign` for
adding/removing channels, close for closing and send(msg, Message.Dispatch)
for upstream events
* `SharedFlowProducer` no longer has a back dependency on
`ChannelManager`, rather it accepts a `suspend (Message.Dispatch) ->
Unit`
* `ChannelManager` is now built with the upstream flow rather than a
flow factory. Given that a flow is stateless, passing in a flow that can
be re-consumed seems like the simpler API.
* New `ChannelManager` tests
* make changes to existing test minimal
* revert actor delegation to reduce PR size
* comments
* clean up tests
* Bump travis
* lint
* rename `channelManagerInbox` ->``sendUpsteamMessage`
* Fix multiple collections on Multicast
Multicast implementation had a bug where the returned flow could
not be collected multiple times as it was using the same channel
it created when was called.
This PR changes it to create per collection to avoid this issue.
I've also replaced function with a field as there
is no reason to keep creating a new one, it can be just a flow
Test: MultiplexTest#multipleCollections
Fixes: #26
* remove create function
* Cleanup StoreBuilder interface and add some documentation
* Update store/src/main/java/com/dropbox/android/external/store4/StoreBuilder.kt
Co-Authored-By: Mike Nakhimovich <digitalbuddha@users.noreply.github.com>
* A little more docs around persistence
* add dokka for store module
move key parser to tests
* remove parser exception
* move RealStore and SourceOfTruth to internal
* move cache type to internal
* move Clearable to tests
* move fetcher to tests
* move multiplexer to internal
* move map indexed to internal
* random cleanup.
Fixed a bunch of warnings.
Moved SimplePersisterAsFlowable into tests as it is not used anymore but
FlowTest uses it.
Moved MemoryPolicyBuilder test to kotlin to make DEFAULT_POLICY a const
* massive file migration
Rename pipeline tests into impl tests
This PR fixes a bug in StoreRealActor where we could call onClose
while actor is processing some messages. Now instead we send a token
to close and inside the message handler we close the channel so that
no new messages can arrive after close meanwhile messages that arrived
before that close is handled properly.
Also set version to 4.0.0 and cleaned up some code.
Fixes#55
* remove suspend cache
This PR replaces suspend cahce usages w/ direct guava cache usage.
I think we should also get rid of guava cache.
First of all it is in java so needs to be moved to kotlin at least for
future multi-platform support.
Second, it would be nicer to have something that uses kotlin's time so
that people can provide a scope w/ a delay functionality (like the
TestCoroutineScope) and also test time related stuff. An alternative might
be letting people pass a Timer in the builder
Fixes#41
* remove commented code :/
This PR implements piggybacking in fetch controller which means even if
upstream closes, we never close downstream flows in fetcher multiplexing.
Instead, we keep them around and if a new request comes in, they'll receive
the same data as well or even keep the upstream open.
This is implemented as a feature of multiplexer as it already has all of the
machinery to do such re-distribution of data.
It is a bit questionable behavior for multiplexer but it is already very
specific to our use case so I thought it might be fine to do it there
* WIP publish test
* add actor based implementation, seems the most promising
* add notes into channel manager as well
* use unlimited channel on the receiver to avoid launching to send
* carry over remaining subscribers into a new flow
* dispatch errors from upstream to all downstreams
* carry over all leftovers at once to avoid starting producer before all is added to the list
* handle swapping channel managers in the consumer
This CL fixes an issue where we wouldn't unsubscribe from the right channel if
the downstream is moved between channel managers due to not receiving any event
after registering.
I've also cleaned up dispatchError to close the channel with error instead of
passing it down as if it is value and throwing again
* allow live buffering
this adds a live buffering functionality to actor publish where
it only buffers if the upstream is still running
* remove logging
* move into src
* code cleanup, more comments
* first shot at new internal store, tests pass, code ugly
* tmp builder for real internal store, starting pipeline tests
* more wip in fixing pipeline store tests, a lot to cleanup
* all tests pass
* code cleanup
* move multiplexer inside store4
* lots of cleanup of unnecessary code
* release barriers that are not used
* don't use pipeline persister.
also fixed a barrier cleanup code in source of truth with barrier
* revert simple persister as flowable change
cleanup for clearstorememorytest
* close multiplexers in fetcher controller when not used
* code style fixes
This CL avoids having two different codepaths based on whether
we skip disk cache or not. It should eventually make it easier
to implement sharing, if we decide to do it here
* Introduce StoreResponse
This CL changes Pipeline API to dispatch StoreResponse instead of Output.
It will allow us to add more metadata into it and also handle things like
source attribution and/or loading state.
This CL only makes the tests pass, does not do anything for actually dispatching
loading state
* first shot at adding a loading state
This CL adds proper StoreResponse to the built in pipelines such
that they can merge information between network and disk.
It needs some cleanup and PipelinePersister's merging logic is
fairly complicated but i couldn't yet find a simpler way to achieve it
* Add more tests to pipeline
Also fixed a bug in cached where it would emit cached value as
Loading even if no refresh is requested
* fix room sample after rebase
* Update the room sample to support refresh
Also some code cleanup in PipelinePersister
* code cleanup
* explain network + disk flow
* Change StoreResponse for plan D
This CL changes store response to act like plan D described here:
https://paper.dropbox.com/doc/Streaming-Data-How-To--AkwrNu0A7\~BikNOP2m20\~R2_AQ-oLBV39GrNimqGDe1vNQMb
Now StoreResponse has 3 types: Data, Error, Loading and each of these have an field.
Loading and Error does not carry over last data anymore, instead, it is expected downstream
to merge them for UI, if desired.
This change makes Store more verbose but i think it is the right direction to allow
developer to expand on it as desired
* update AGP to 3.6-alpha11
* update pipeline store tests to check finite streams
This CL updates PipelineStoreTest to consume the whole stream
when we expect it to finish rather than limiting the number of
dispatched items
* update RoomActivity to respect new StoreResponse
* add search to room activity
* code cleanup
