Commit graph

611 commits

Author SHA1 Message Date
Burak Eregar
ee30afb0c9 Implement Truth and migrate store module. (#30) 2019-12-15 16:30:34 -05:00
Yigit Boyar
2b70a2f721 Fix multiple collections on Multicast (#28)
* Fix multiple collections on Multicast

Multicast implementation had a bug where the returned flow could
not be collected multiple times as it was using the same channel
it created when  was called.

This PR changes it to create per collection to avoid this issue.

I've also replaced  function with a  field as there
is no reason to keep creating a new one, it can be just a flow

Test: MultiplexTest#multipleCollections
Fixes: #26

* remove create function
2019-12-13 13:08:35 -05:00
Eyal Guthmann
fa10e84623
Cleanup StoreBuilder interface and add some documentation (#24)
* Cleanup StoreBuilder interface and add some documentation

* Update store/src/main/java/com/dropbox/android/external/store4/StoreBuilder.kt

Co-Authored-By: Mike Nakhimovich <digitalbuddha@users.noreply.github.com>

* A little more docs around persistence
2019-12-11 14:52:11 -08:00
David Algar
98062b155b Update Readme.md with correct artifact id (#23)
The artifact id `store4`, so the current included example dependency line doesn't locate the library.   Changing it to `store4` correctly finds the library in the local repository.
2019-12-11 11:53:33 -05:00
Gert Dingenen
9f7fd1bd17 Replace Retrofit's deprecated Deferred CallAdapter with its built-in suspend support. (#22) 2019-12-11 10:51:48 -05:00
ychescale9
b4c0877732 Fix a bunch of kotlin compiler warnings. (#20) 2019-12-11 09:44:45 -05:00
Mike Nakhimovich
7cb52099f6
Update README.md 2019-12-10 20:46:08 -05:00
ychescale9
1810355341 Update dependencies and clean up unused ones (#16)
* Update dependencies and clean up unused ones.

* Update okio to 2.4.1 and migrate deprecated APIs to extension functions.

* Update platform and build-tools in travis config.
2019-12-10 20:12:51 -05:00
Mike Nakhimovich
02fa6facbe
We are using travis for ci (#17) 2019-12-10 16:10:19 -05:00
Mike Nakhimovich
ff06e28697
Update readme to trigger travis build (#15)
* Update readme to trigger travis build

* fix deploy script to not refer to nytimes

* Update deploy_snapshot.sh
2019-12-10 13:48:43 -05:00
David Chang
a19a9be424 Update Travis secure variables (#18) 2019-12-10 12:43:40 -05:00
ychescale9
8ddbe3c76f Add ktlint and fix all issues. (#11) 2019-12-10 10:51:14 -05:00
David Chang
dd5a5c8a78
Merge pull request #13 from dropbox/changusmc-patch-1
Update README with Store maven gradle dependency
2019-12-09 12:59:05 -08:00
David Chang
03c7ef6267
Update README with Store maven gradle dependency 2019-12-09 12:11:19 -08:00
David Chang
93112069a5
Merge pull request #12 from changusmc/master
Update maven coordinates for Store 4
2019-12-09 11:43:26 -08:00
David Chang
dce8b865c1 Update maven coordinates for Store 4 2019-12-09 11:39:20 -08:00
Yigit Boyar
6d97c399fa move multicast to its own module (#9)
Fixes #8
2019-12-08 19:12:11 +01:00
Mike Nakhimovich
a362c33c0a
Remove parsing blurb from Readme (#4)
Store3 took a parser, store4 doesn't need to due to how easy it is to add transformations to the fetcher or reader flows
2019-12-07 00:51:29 +01:00
Mike Nakhimovich
088be08376
Update license to include Dropbox copyright (#7) 2019-12-07 00:51:17 +01:00
Mike Nakhimovich
fed4430397
Create pull_request_template.md 2019-12-07 00:49:00 +01:00
Mike Nakhimovich
1314546bb9
Update CONTRIBUTING.md 2019-12-07 00:30:10 +01:00
Mike Nakhimovich
59ab3f0889
Add not about missing maven artifact (#6)
Add how to build from source and install into local m2 repo
2019-12-07 00:19:36 +01:00
Júlio Zynger
5d6ccf8422 Fix link in README (#3)
* Fix link in README

* Oops bracket
2019-12-07 00:10:40 +01:00
Yigit Boyar
2e80c6d7df Update readme (#2)
* update readme to show new streaming APIs

* more readme fixes, styles etc

* fix version to snapshot
2019-12-06 16:59:07 +01:00
William Clausen
2cc626013d
Merge pull request #1 from yigit/google-header
add missing google copyright header
2019-12-05 15:32:52 -08:00
Yigit Boyar
1a750f54db add missing google copyright header 2019-12-06 00:28:31 +01:00
Mike Nakhimovich
4fa9c252b6
rename package from nytimes to dropbox (#63)
* rename package from nytimes to dropbox

* Update build.gradle

Co-Authored-By: David Chang <changd@gmail.com>
2019-12-05 19:52:47 +01:00
Yigit Boyar
5816adabe5 Cleanup (#62)
* add dokka for store module

move key parser to tests

* remove parser exception

* move RealStore and SourceOfTruth to internal

* move cache type to internal

* move Clearable to tests

* move fetcher to tests

* move multiplexer to internal

* move map indexed to internal
2019-12-04 09:40:29 +01:00
Yigit Boyar
b12aad7d2d
store fluent API prototype (#60)
StoreBuilder API that does not require excessive type arguments.
2019-12-02 14:30:45 -08:00
Yigit Boyar
3b44944fc5 Rename Multiplex to Multicast (#59)
Appearantly, multiplex is N to 1 while this thing is 1 to N. Yigit should consider checking what things mean before using them in code ¯\_(ツ)_/¯
2019-11-27 20:55:19 -05:00
Yigit Boyar
d5e7f196d1 faster builds (#58)
enable build cache
enable configure on demand
remove snapshot repo from repositories as we never use it
but it is very slow
2019-11-23 21:34:10 -05:00
Yigit Boyar
52aefe0094 random code cleanups (#57)
* random cleanup.

Fixed a bunch of warnings.
Moved SimplePersisterAsFlowable into tests as it is not used anymore but
FlowTest uses it.
Moved MemoryPolicyBuilder test to kotlin to make DEFAULT_POLICY a const

* massive file migration
Rename pipeline tests into impl tests
2019-11-23 14:00:34 -05:00
Yigit Boyar
c223e44d8d
Close channels w/ a suspend function. (#56)
This PR fixes a bug in StoreRealActor where we could call onClose
while actor is processing some messages. Now instead we send a token
to close and inside the message handler we close the channel so that
no new messages can arrive after close meanwhile messages that arrived
before that close is handled properly.

Also set version to 4.0.0 and cleaned up some code.

Fixes #55
2019-11-20 07:11:31 -08:00
Yigit Boyar
8347de79aa Remove suspend cache (#52)
* remove suspend cache

This PR replaces suspend cahce usages w/ direct guava cache usage.
I think we should also get rid of guava cache.
First of all it is in java so needs to be moved to kotlin at least for
future multi-platform support.
Second, it would be nicer to have something that uses kotlin's time so
that people can provide a scope w/ a delay functionality (like the
TestCoroutineScope) and also test time related stuff. An alternative might
be letting people pass a Timer in the builder

Fixes #41

* remove commented code :/
2019-11-14 07:31:07 -05:00
miken
b7a4c89cc0 uml diagram 2019-11-12 11:16:10 -05:00
Yigit Boyar
42c37d7656 Spotless integration (#51)
* apply spotless for unified code style

* configure circleci to run spotless
2019-11-09 17:02:01 -05:00
Yigit Boyar
3e84b6bdb5
Stream without persister (#48)
This PR implements piggybacking in fetch controller which means even if
upstream closes, we never close downstream flows in fetcher multiplexing.
Instead, we keep them around and if a new request comes in, they'll receive
the same data as well or even keep the upstream open.

This is implemented as a feature of multiplexer as it already has all of the
machinery to do such re-distribution of data.
It is a bit questionable behavior for multiplexer but it is already very
specific to our use case so I thought it might be fine to do it there
2019-11-09 08:18:11 -08:00
Mike Nakhimovich
1356a27975
clean up readm (#45) 2019-11-04 21:11:07 -05:00
Mike Nakhimovich
b14b307c2c
Miken/kill2 (#43)
* cleanup with no explanation

* collapse interfaces to only Store, remove parsers

* collapse interfaces to only Store, remove parsers
2019-11-03 15:06:12 -05:00
Mike Nakhimovich
ca8c957a2d
cleanup with no explanation (#42) 2019-11-03 14:53:38 -05:00
Yigit Boyar
ba22d0c66c Internal Store 3rd attempt (#35)
* WIP publish test

* add actor based implementation, seems the most promising

* add notes into channel manager as well

* use unlimited channel on the receiver to avoid launching to send

* carry over remaining subscribers into a new flow

* dispatch errors from upstream to all downstreams

* carry over all leftovers at once to avoid starting producer before all is added to the list

* handle swapping channel managers in the consumer

This CL fixes an issue where we wouldn't unsubscribe from the right channel if
the downstream is moved between channel managers due to not receiving any event
after registering.

I've also cleaned up dispatchError to close the channel with error instead of
passing it down as if it is value and throwing again

* allow live buffering

this adds a live buffering functionality to actor publish where
it only buffers if the upstream is still running

* remove logging

* move into src

* code cleanup, more comments

* first shot at new internal store, tests pass, code ugly

* tmp builder for real internal store, starting pipeline tests

* more wip in fixing pipeline store tests, a lot to cleanup

* all tests pass

* code cleanup

* move multiplexer inside store4

* lots of cleanup of unnecessary code

* release barriers that are not used

* don't use pipeline persister.
also fixed a barrier cleanup code in source of truth with barrier

* revert simple persister as flowable change
cleanup for clearstorememorytest

* close multiplexers in fetcher controller when not used

* code style fixes
2019-11-03 10:41:32 -05:00
Yigit Boyar
e9d6a91d83 Merge PersisterPipeline code path (#34)
This CL avoids having two different codepaths based on whether
we skip disk cache or not. It should eventually make it easier
to implement sharing, if we decide to do it here
2019-10-02 13:24:36 -04:00
Yigit Boyar
915bd7d4d7 StoreResponse (#33)
* Introduce StoreResponse

This CL changes Pipeline API to dispatch StoreResponse instead of Output.
It will allow us to add more metadata into it and also handle things like
source attribution and/or loading state.

This CL only makes the tests pass, does not do anything for actually dispatching
loading state

* first shot at adding a loading state

This CL adds proper StoreResponse to the built in pipelines such
that they can merge information between network and disk.

It needs some cleanup and PipelinePersister's merging logic is
fairly complicated but i couldn't yet find a simpler way to achieve it

* Add more tests to pipeline

Also fixed a bug in cached where it would emit cached value as
Loading even if no refresh is requested

* fix room sample after rebase

* Update the room sample to support refresh

Also some code cleanup in PipelinePersister

* code cleanup

* explain network + disk flow

* Change StoreResponse for plan D

This CL changes store response to act like plan D described here:
https://paper.dropbox.com/doc/Streaming-Data-How-To--AkwrNu0A7\~BikNOP2m20\~R2_AQ-oLBV39GrNimqGDe1vNQMb

Now StoreResponse has 3 types: Data, Error, Loading and each of these have an  field.
Loading and Error does not carry over last data anymore, instead, it is expected downstream
to merge them for UI, if desired.

This change makes Store more verbose but i think it is the right direction to allow
developer to expand on it as desired

* update AGP to 3.6-alpha11

* update pipeline store tests to check finite streams

This CL updates PipelineStoreTest to consume the whole stream
when we expect it to finish rather than limiting the number of
dispatched items

* update RoomActivity to respect new StoreResponse

* add search to room activity

* code cleanup
2019-09-22 11:23:59 -04:00
Yigit Boyar
567524380f
Add sample with room (#32)
This sample uses the new pipeline API directly.
To avoid changing Model.kt, the PostEntity is a weird wrapper
but i think it is fine.
2019-08-31 15:33:42 -07:00
Yigit Boyar
d27fb96384
update coroutines to flow stable (#31)
This PR updates coroutines to 1.3.0, kotlin to 1.3.50 and agp to 3.6.alpha08.

I've also cleaned all usage of FlowPreview
2019-08-27 18:43:48 -07:00
Yigit Boyar
92022be61a
let persister flow nullables to understand absense. (#30)
This is possibly temporary until we get a StoreResponse but may still keep
it as we wouldn't want to force everything to support StoreResponse
2019-08-15 23:24:21 +03:00
Yigit Boyar
d1ac07d8d0
Update the sample to use LifecycleScope and ListAdapter (#29) 2019-08-15 23:22:11 +03:00
Mike Nakhimovich
d86c6573da
Yigit fixes another thing (KeyTracker) (#27)
* Updated coroutines to RC02 and fix Flow invariance

This PR fixes an invariance problem in side collect where it was
creating a scope and emitting from there.

* Fix KeyTracker not to lose emissions

KeyTracker would lose some emissions if they happen in rapid succession
between different keys. This new implementation maintains a separate
channel for each key to avoid that. Since we now create a channel per key,
we also need to track and clean them up to avoid consuming too much memory
2019-08-11 21:34:18 -04:00
Mike Nakhimovich
80e834d536
remove clearMemory, upgrade coroutines (#20)
* remove clearMemory, upgrade coroutines

* rebase, pr comments

* fix build

* fix build

* convert more tests to testScope

* few more conversions
2019-08-11 21:27:36 -04:00
Yigit Boyar
9d8aec74e3 Updated coroutines to RC02 and fix Flow invariance (#25)
This PR fixes an invariance problem in side collect where it was
creating a scope and emitting from there.
2019-08-11 07:49:32 -04:00