Commit graph

198 commits

Author SHA1 Message Date
lukisk
4cc5b052d9
release fetcher in nonCancellable scope (#432)
* release fetcher in nonCancellable scope

* replace wildcard imports
2022-04-22 19:20:25 -05:00
Sam Edwards
83205da5ec
Version Bumps - AGP 7.1.2, Kotlin 1.6.10 and related updates (#414)
* Latest is 0.8.0, but 0.4.0 works.  Starts to break at 0.5.0.

* Kotlin Atomic Plugin working with 0.14.4

* Atomic Kotlin Plugin working with 0.15.0

* Atomic Kotlin Plugin working with 0.15.1

* Upgrade Gradle Wrapper to 7.4.1

* AGP 4.1.0 -> 4.2.0

* AGP 4.2.0 -> 7.0.2

* AGP 7.0.2 -> 7.1.2

* Kotlin 1.6.10 and Spotless Plugin 6.3.0 Upgrade

* WARNING:The specified Android SDK Build Tools version (29.0.3) is ignored, as it is below the minimum supported version (30.0.3) for Android Gradle Plugin 7.1.2.

* Using --rerun-tasks instead of clean.

* KT Lint formatting update.

Co-authored-by: Sam Edwards <samedwards@dropbox.com>
2022-03-17 14:03:15 -04:00
David Chang
bf2adbf127
Move to Maven Publish Gradle Plugin (#358)
* Move to Maven Publish Gradle Plugin

* Remove release scripts

* Update release docs
2021-12-08 12:45:33 -05:00
Marco Romano
3f71c43d99
Revert to Duration.INFINITE (#353)
This should still maintain compatiblity with Kotlin 1.4 and 1.5 whilst also ensuring forward compatiblilty with Kotlin 1.6.
Compatibility with 1.4-M3 should be safe to drop.
2021-12-07 09:52:21 -05:00
Chris Mitchell
862da9b4f9
Fix missing experimental annotations (#305)
* Revert "Remove unnecessary FlowPreview & ExperimentalCoroutinesApi annotations.  These are not needed with newer versions of Kotlin. (#289)"

This reverts commit 61b8cb0633.

* Fix missing experimental annotations
2021-09-09 08:46:48 -07:00
Sam Edwards
61b8cb0633
Remove unnecessary FlowPreview & ExperimentalCoroutinesApi annotations. These are not needed with newer versions of Kotlin. (#289) 2021-09-08 18:57:02 -04:00
Yigit Boyar
e79ae4362f
Update kotlin to 1.5.21, coroutines to 1.5.1 (#281)
This is just a version bump, we still need to followup
with cleanup.

Test: ./gradlew check
2021-07-25 15:15:32 -07:00
Mike Nakhimovich
86ab7a5251
Fire off kotlin 1.5 compatible snapshot (#273)
* upgrade to kotlin 1.5 warning breaking change

* upgrading to kotlin15, create new snapshot

Co-authored-by: miken <miken@dropbox.com>
2021-05-10 14:28:52 -04:00
Mike Nakhimovich
35fd9af24f
Remove deprecated code to fix error when users upgrade to kotlin 1.5 (#265)
* upgrade to kotlin 1.5 warning breaking change

* achieve compatibility with 1.4 and 1.5

* trigger travis

* fix lint

Co-authored-by: miken <miken@dropbox.com>
2021-05-06 11:45:17 -04:00
Mike Nakhimovich
7ba28a8588
add piggyback to all stores (#257)
* add piggyback to all stores

* lint

* pr feedback

Co-authored-by: miken <miken@dropbox.com>
2021-02-04 02:26:25 -05:00
Yigit Boyar
d6e73e1fe2
update coroutines to 1.4.0, kotlin to 1.4.10 (#242)
* update coroutines to 1.4.0, kotlin to 1.4.0

This CL updates coroutines to 1.4.0, kotlin to 1.4.10 and
also removes the only usage of ConflatedBroadcastChannel
(in SOTWithBarrier) with MutableStateFlow.

* expand SOTWithBarrier failure test

This change improves the failure test case to ensure it continues
for future writes
2020-11-08 14:52:26 -08:00
Eyal Guthmann
e773021fec
Revert to use the old guava port cache (via jar) (#200)
* Revert to using guava port cache by depending on jar

* Apply suggestions from code review

* revert code to old cache

* clean up build.gradle

* manually remove cache4 entries that are regenerated locally because of some dirty cache?

* bump gradle

* rerun tasks

* try to deflake rx tests

* cache builder tests

* api dump

* bump ktlint to work with 1.4 features

* formatting

Co-authored-by: Mike Nakhimovich <digitalbuddha@users.noreply.github.com>
2020-09-14 14:49:34 -07:00
ghus-raba
8c8fcc8271
Move filesystem types (#229)
* Move Persister to filesystem module

* Move DiskRead to filesystem module

* Move DiskWrite to filesystem module

Co-authored-by: Frantisek Nagy <frantisek.nagy@erstegroup.com>
2020-09-11 08:35:51 -07:00
ghus-raba
6002efdd90
Replace BarCode with Pair<String, String> (#228)
Co-authored-by: Frantisek Nagy <frantisek.nagy@erstegroup.com>
2020-09-10 08:56:23 -07:00
Eyal Guthmann
1a4b0b0966
use covariance to remove generic types from subtypes like Error (#220) 2020-08-21 14:08:38 -07:00
Yang
a0477c1ed2
Update to Kotlin 1.4.0, Coroutines 1.3.9 and revert to using Duration.INFINITE (#195)
* Update to Kotlin 1.4.0-rc and Coroutines 1.3.8-1.4.0-rc, fix type inference issues.

* Kotlin 1.4.0.

* Coroutines 1.3.9.
2020-08-14 14:17:17 -04:00
Eyal Guthmann
ecc3470d55
Add StoreResult.NoNewData for empty fetchers. (#194)
* Add StoreResult.NoNewData for empty fetchers.

* handle no SoT case

* update comments
2020-08-07 09:26:51 -07:00
Eyal Guthmann
eb5b8c2fe8
Fix memory leak caused by capturing the user's coroutine context when… (#199)
Fix memory leak caused by capturing the user's coroutine context when creating a fetcher
2020-08-05 15:37:51 -07:00
Orhan Tozan
868059d060
Remove duplicate comment (#192) 2020-07-28 11:58:22 -07:00
Eyal Guthmann
b18db6f61f
Minor cleanup in RealStore.diskNetworkCombined (#190)
* Minor cleanup in RealStore.diskNetworkCombined

* add missing comment

* one more if-else -> when

* lint
2020-07-23 16:30:57 -07:00
Chris Banes
9c9f70a16d
Avoid using Duration.INFINITE (#189)
It currently breaks consumers compiling with Kotlin
1.4-M3 (not sure why).

Added some TODOs to revert back to using Duration.INFINITE
once Store is compiled with Kotlin 1.4

Closes #188
2020-07-23 09:08:10 -07:00
Yigit Boyar
61762f2f96
Emit source of truth errors to subscribers #177 (#180)
* reproduce #177 and fix reader errors

* dispatch write errors to receiver

Now source of truth can unblock reader while also letting them
know that an error happened while writing. This forces readers
to first dispatch the error then whatever data they have.

I've also added new public WriteException/ReadException classes
to SourceOfTruth so that it is easy to diagnose these problems
when it hits to the developer's code
2020-06-27 11:57:04 -07:00
Yigit Boyar
b86050496c
Yigit/move fetcher factories to fetcher (#168) (#181)
* Move Fetcher factories into companion

Fetcher factories were global methods, which made them hard
to discover since IDE cannot easily auto-complete.

This PR moves them into the companion of Fetcher while also
making Fetcher a real interface instead of a typealias.

Even though it is a bit more code for the developer, now they
can easily discover how to create a Fetcher by typing Fetcher.

Fixes: #167

* make rx methods start w/ from too for consistency

* Rename fether factories to be more clear, hopefully :/

* remove fetch method, use invoke instead

* Make Fetcher.from the one that receives a suspend fun.
Create Fetcher.fromFlow for the flowing version.

Rename both SourceOfTruth builder methods to . Rely on param names to disambiguate

* use .of instead, this seems better to me.

We should probably get rid of StoreBuilder.from and make it
Store.builder()

* fix jvm name for SourceOfTruth.of with flow function

* fix RxSourceOfTruth name to match original class

* specify bounds for FactoryFetcher

* updates per PR review

* update graph per SoT rename

* update rxjava3 APIs as well

These appeared after i rebased, missed them completely.
Also fixed some tests, appearantly IJ parameter name refactor does
not always work

* supress wrong unnecessary cast warning

without this, multicaster cannot resolve to the base StoreResponse type

* upgade gradle, try to fix build by disabling caching

* split subscribers

* resubscribe

Co-authored-by: miken <miken@dropbox.com>

Co-authored-by: miken <miken@dropbox.com>
2020-06-19 06:00:50 -04:00
Yuki312
a5c29ffd3e
fix doc: default policy of max items = 100 (#169) 2020-05-30 16:47:44 -07:00
Yang
7c3ce55b63
Coroutines 1.3.5 (#166)
* Update Kotlin, Coroutines, Rx2, OkHttp, Okio, Gradle.

* Clean up @FlowPreview @ExperimentalCoroutinesApi which are no longer required.
2020-05-09 10:21:25 -04:00
Eyal Guthmann
e71a4e9e4a
Support non exception errors from fetcher (#123)
* Support non exception errors from fetcher

* revert parital changes to store builder to reduce noise

* finish off diff

* Allow to create a FetcherResult.Error without a Throwable. Add tests

* Add missing funcion and more tests

* lint

* unflake RxFlowableStoreTest

* try to rename FakeFetcher to FakeRxFetcher to (maybe) solve missing codcov

* move SourceOfTruth out of impl package

* Rename accidental change of RxStoreBuilder.fromMaybe back to formSingle

* Introduce Fetcher from #139

* fix Rx artifact

* delete legacy presistor factory

* fix api file

* move fetcher to be a typealias

* code review comments + clean up documentation

* code review comments

* Update store/src/main/java/com/dropbox/android/external/store4/Fetcher.kt

Co-Authored-By: Yigit Boyar <yboyar@google.com>

* Revert "Update sample app's build.gradle to refer to the externally released version of Store (#159)"

This reverts commit fc8da86f63.

* update releasing.md

Co-authored-by: Yigit Boyar <yboyar@google.com>
2020-04-29 10:31:10 -07:00
Dave Parker
8f0625fc70
Contain @ExperimentalStdlibApi within relevant scope. (#154)
* Contain @ExperimentalStdlibApi within relevant scope.

The alpha04 release of Store caused the requirement of the @ExperimentalStdlibApi annotation for client code. I looked into it, and the only thing in Store that required the annotation was the Kotlin ArrayDeque used in the BufferImpl class. That class is private in the ChannelManager.kt file, so I opted that file into the annotation and removed all other occurences of the annotation.

Code compiles, all 208 tests pass.

* update to correct git glitch in /build.gradle

* update to address 'needless blank line' linter

* Prepare for release 4.0.0-alpha05

* Prepare for next development version.

Co-authored-by: Dave Parker <davidp@dropbox.com>
2020-04-19 19:51:50 -04:00
Yang
e6158669d4
Fix a deadlock during cache eviction (#155)
* Fix a deadlock during cache eviction due to duplicate CacheEntries with different accessTime being added to accessOrder HashSet.

* Rename test.
2020-04-19 19:50:12 -04:00
Claus Holst
ee133dc5a0
#126 [Feature Request] Migrate cache library to use kotlin.time APIs (#129)
* Migrate cache library to use kotlin.time APIs

* Fix docs and inline durations in tests

* Migrate cache library to use kotlin.time APIs

* Only require ExperimentalTime when setting cache expiration

* Set ExperimentalTime on RealStore to use it on MemoryPolicy

* Set ExperimentalTime on Cache interface and not on functions

* Avoid ExperimentalTime annotation in filesystem module

* Remove unneeded ExpirationTime annotation

* Remove unneeded ExpirationTime annotation
2020-03-16 14:29:19 -07:00
Claus Holst
5d5e89b13b
Use Kotlin version of ArrayDeque in ChannelManager (#134) 2020-03-16 08:08:32 -07:00
Yang
38baeb7c8b
Kotlin 1.3.70 and other dependencies updates. (#125)
* Update Kotlin, Gradle, binary-compatibility-validator, AGP, AndroidX, retrofit, okhttp, rx2.

* Replace deprecated @UseExperimental with @OptIn.

* Gradle 6.2.2.

* Update Coroutines to 1.3.4. Fix more compiler warnings.

* Update build-tools for travis.
2020-03-07 09:38:02 -05:00
Yigit Boyar
6b1d5c1965
Ensure network starts after disk is established (#115)
* Ensure network starts after disk is established

This PR fixes a bug in the ordering where if disk is skipped and network
is super fast, there is a tiny chance that we wouldn't latch onto source
of truth before triggering network and that would make SourceOfTruth think
that the value was written to disk before it observed, hence become a disk
value rather than fetcher.

This PR ensures that we unlock the network only after we establish a position
in the disk read queue so that we'll know the fetcher value came afterwards.

Fixes: 114
Test: RxSingleStoreTest

* Apply suggestions from code review

* move network unlock to disk flow setup for clarity

Co-authored-by: Mike Nakhimovich <digitalbuddha@users.noreply.github.com>
2020-02-18 19:56:37 -08:00
Mike Nakhimovich
9111d667c5
more tests! (#106)
* more tests!

* Start tracking API binary compatibility (#97)

* Configure binary-compatibility-validator plugin.

* Generate current API dumps.

* Update CONTRUBUTON.md with instruction for updating API dumps.

* ktlint fix

* newlines

* put fs eraser back in

* more tests!

* ktlint fix

* newlines

* put fs eraser back in

* run api checks

* lint

* lint

Co-authored-by: Yang <ychescale9@gmail.com>
2020-02-16 08:35:35 -05:00
Yang
8bc3c76eb1
Start tracking API binary compatibility (#97)
* Configure binary-compatibility-validator plugin.

* Generate current API dumps.

* Update CONTRUBUTON.md with instruction for updating API dumps.
2020-02-15 11:59:42 -05:00
David Chang
536af1f455
Add code coverage (Jacoco) and CodeCov (#100)
* [WIP] Add code coverage

* Get multi-module and merging working

* Add CodeCov

* Update location of report.xml

* Check instead of build depends on Jacoco

* Do not merge coverage

* Ignore tests

* update comment
2020-02-13 17:42:05 -08:00
Mike Nakhimovich
7b7f01a439
Rxbindings (#93)
RxJava2 extensions
2020-02-12 20:36:10 -05:00
Yang
613a5c8296 Add support for purging all store entries (#79)
* Fix typos.

* Gradle 6.1, AGP 4.0.0-alpha09, lifecycle 2.2.0.

* Add experimental invalidateAll() support and @ExperimentalStoreAPI annotation.

* Update sample with deleteAll function.

* Update README.md with deleteAll function.

* Add a section to README.md on clearing store entries.

* Rewrite tests without mocking. Move test utils / helpers to a single package.

* Code formatting and cleanups.

* Use StoreResponse.Data instead of DataWithOrigin in ClearAllStoreTest and ClearStoreByKeyTest.

* Simplified samples. Refactor tests.

* Gradle 6.1.1.
2020-01-28 09:02:28 -08:00
Eyal Guthmann
86f7b1b060
introduce piggyback only downstreams to multicaster and fix #59 (#75)
* patched-from-yigit

* piggyback only channels

* cleanup docs and tests

* add yigit's test. piggyback now acks upstreams, pr comments

* tests names

* fix test name

* comments

* Update store/src/test/java/com/dropbox/android/external/store4/impl/FlowStoreTest.kt

Co-Authored-By: Yigit Boyar <yboyar@google.com>

* remove import

Co-authored-by: Yigit Boyar <yboyar@gmail.com>
2020-01-25 13:13:12 -08:00
Yigit Boyar
929b4c40a4 Change flow collection util to drain the flow (#64)
* Change flow collection util to drain the flow

This PR changes the assertItems util method in tests to drain the
flow to ensure we don't produce unwated items.

* reformat code

* Use a test subject for flow assertions

* update test subject messsages, add tests for it (so meta :p)

* Apply suggestions from code review

Co-Authored-By: Eyal Guthmann <eyalgu@users.noreply.github.com>

* fix code style

* Don't try to get exception unless collection finished

This fixes a bug in FlowSubject where it would try to get the cancelation
exception from an unfinished job. Also added a test.

Co-authored-by: Eyal Guthmann <eyalgu@users.noreply.github.com>
2020-01-19 10:56:16 -08:00
ychescale9
c29ed5583d Fix an inconsistency between Store's MemoryPolicy and Cache where the default MemoryPolicy created has a maxSize of 1. (#71) 2020-01-18 09:25:05 -05:00
Eyal Guthmann
5f2fb35da9
Avoid illegal cast in RealStore.stream (#69)
* Avoid illegal cast in

* remove commented out code

* remove wildecards

* ktlint

* optimize imports

* don't print  in exception message to avoid PII leak
2020-01-17 16:29:17 -08:00
Stephen Bain
bc6c47833d added docs to MemoryPolicy.setMemorySize (#67) (#68) 2020-01-17 16:45:58 -05:00
ychescale9
7ee32a7468 Clean up build configs (#62)
* Update dependencies, clean up unused dependencies.

* Remove unused checkstyle and pmd rulesets.

* Use `plugins` block.

* Fix compiler warnings.
2020-01-13 12:43:41 -05:00
ychescale9
062deda898 Cache Rewrite (#49)
* Rewrite cache module in Kotlin.

* Migrate RealStore to use new cache.

* Migrate FileSystemImpl to use new cache.

* Migrate StoreTest to new cache.

* Remove Guava Cache.

* Migrate remaining Java file to Kotlin.

* Fix ktlint error.

* Fix package.

* Add support for CacheLoader.

* Remove unnecessary mockito-inline.

* Remove volatile keyword from CacheEntry fields and add docs re. cache entry reuse and thread-safty assumption.

* - Refactor CacheBuilder.
- Update docs.
- Update Key, Value, Input, Output generic types to be bound by Any.
- Replace spy with TestLoader for asserting invocation.

* Support key-based synchronization for get(key: Key, loader: () -> Value): Value.

* Document loader exception propagation.

* Small clean ups.

* Make CacheEntry's Key and Value bound to Any, move CacheEntry to RealCache.kt as a private class.

* Add reference counting key-based synchronization for cache loader.

* Improve KeyedSynchronizer and docs.

* Refactor tests and add a missing test case to reach 100% coverage.

* Add memory barrier after updating access time in cache entry.

* Only process access queue if expiresAfterAccess == true.

* Support configurable concurrency level.

* Increase delay for concurrency test.

* Explicitly set concurrency level for concurrency tests.

* Only add memory barrier after recordRead if expiresAfterAccess is enabled.

* Fix a potential memory leak caused by thread preemption.

* Allow retries (3 times) for concurrency tests.

* Move performMaintenance function to bottom of the class.

* Fix typos, rename performMaintenance() to cleanUpDeadEntries(), improve docs.

* Fix more typos.

* Only remove entries in map but not in queue.

* Replace memory barrier with @Volatile fields in CacheEntry.

* Fix deadlock while iterating sets.

* Remove stale cache/README.md, remove commented out java file.
2020-01-08 06:56:02 -05:00
Mike Nakhimovich
267f803db0
fix dokka, update comments, integrate persister (#56)
* fix dokka, update comments, integrate persister
2020-01-07 14:08:53 -05:00
ychescale9
0cd21be2ee Switch POM packaging format to jar for cache, filesystem and store. (#54) 2020-01-07 13:31:19 -05:00
Eyal Guthmann
54c7ad5113
Add an optional param to Multicaster to keep the fetches alive even if all downstreams are closed. (#40) 2020-01-02 13:14:18 -08:00
ychescale9
ba3bf9fd2e Misc improvements and cleanups (#47)
* Add kotlin plugin to cache module

* Remove custom accessors in StoreDefaults as default values are effectively constant.

* Update JUnit to 4.13 RC2 to enable  asserting specific error message on expected exception.

* Remove custom accessors in MemoryPolicy as all values are calculated from constants.

* Remove checkstyle and pmd as the codebase will be 100% Kotlin.

* Fix IDE warning in build.gradle by removing static imports.

* Remove unused guava dependencies.

* Improve StoreDefaults docs.
2019-12-26 13:21:52 -05:00
Pavlos-Petros Tournaris
6827619744 [Kotlinization] Misc refactors for more Kotlin friendlier code (#37)
* Make MemoryPolicy Kotlin friendlier

* Make MemoryPolicy usage more Kotlin like

* Format code in FSReader

* Format code in FSWriter

* Format code in FileSystemRecordPersister

* Format code in FileSystemPersister

* Convert BarCodeReadAllPathResolver to Object instead of Class

* Convert BarCodePathResolver to Object instead of Class

* Fix wrong control flow conversion
2019-12-18 10:36:36 -08:00
Eyal Guthmann
689812a7bb
Clean up ChannelManager's interface to make it easier to add more tests (and add some) (#29)
* Clean up ChannelManager's interface to make it easier to add more tests (and add them)

Main changes:
* ~`ChannelManager` no longer inherits from `StoreRealActor` but rather
delegates to one.~ - Moved to followup PR
* messages coming from the upstream were placed under `Message.Dispach`
rather than `Message`. These are the only messages accepted from outside
`ChannelManager`
* `ChannelManager` now only exposes `plusAsign`, `minusAsign` for
adding/removing channels, close for closing  and send(msg, Message.Dispatch)
for upstream events
* `SharedFlowProducer` no longer has a back dependency on
`ChannelManager`, rather it accepts a `suspend (Message.Dispatch) ->
Unit`
* `ChannelManager` is now built with the upstream flow rather than a
flow factory. Given that a flow is stateless, passing in a flow that can
be re-consumed seems like the simpler API.
* New `ChannelManager` tests

* make changes to existing test minimal

* revert actor delegation to reduce PR size

* comments

* clean up tests

* Bump travis

* lint

* rename `channelManagerInbox` ->``sendUpsteamMessage`
2019-12-16 08:50:57 -08:00