unexpected: AngularFirestore#collection() returns cached value on initial subscription

Version info

Angular: 7.2.5
Firebase: 5.8.3
AngularFire: 5.1.1
Other (e.g. Ionic/Cordova, Node, browser, operating system): node 10.14.2, macOs X

How to reproduce these conditions

I've had some trouble isolating the exact problem. Basically, I have a people collection. If I fetch an individual document from the people collection (lets call it the person A doc), and then subscribe to the entire people collection, I'm seeing that the collection('people').valueChanges() subscription immediately returns an array that just contains person A. The subscription then fires again shortly later with the full results of the person collection.

If I invoke collection('people').valueChanges() before fetching an individual document from the people collection, then collection('people').valueChanges() returns the correct results on the first try.

What I think is happening, is that collection('people') is immediately returning results from an in-memory cache and then updating those results when it hears back from the server. I could see scenarios where this behavior would be desirable, but it is not desirable in my case, was unexpected and hard to diagnose, and there doesn't seem to be an option to disable it. I'm not sure if this is intended behavior or a bug.

I think it's possible that issue #1975 is being caused by the same problem.

In my case, the problem with this behavior (other than the fact that it is unexpected) is that I want to take(1) from the collection('people').valueChanges() query, except now the return value of that action is indefinite.

Debug output

I have none. I did try to set firebase.database().enableLogging(true); as suggested, but this.angularFirestore.app.database() does not have an enableLogging() function.

Expected behavior

By default, I expected query results to not be returned unless they accurately reflect the query.

I could see value in returning results immediately from a cache, but I don't think this behavior should be the default as it's unexpected, hard to diagnose, and can lead to unwanted behavior.

Actual behavior

AngularFirestore#collection() seems to return results immediately from the cache and then update the results when it hears back from the server.

I have been hit by this very badly. In my opinion it is a bug.

example:

you have collection where you have following docs:
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

If you have open subscription to item no 9 and you make query where you limit to 5 first items with take(1) you will receive item [9] instead of [0, 1, 2, 3, 4].

Sample project: https://angular-f6nfor.stackblitz.io

This makes e.g. pagination queries unreliable. When you relay that last item received will be last item of your previous pagination query and you use this item as startAfter for next query.

This is how the Firestore JS SDK works. I'm considering an API change that would allow you to inspect the metadata and say filter { snap -> !snap.metadata.fromCache } on snapshotChanges()

Experiencing this bug too!

Anything would help! Glad someone brought up my old issue from a while ago.

@tmk1991

@tmk1991 How did overcome the problem all these while?

I used a worked around...

you have collection where you have following docs:
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

If you have open subscription to item no 9 and you make query where you limit to 5 first items and explicitly with a where clause that says item that is not previously projected.

Put another way, query with exclusion to not include no 9.

I use the get() call for one of the queries and snapshotChanges() for the other.

@jamesdaniels

@jamesdaniels I can't speak for everyone, but your proposed update would address the problems I am experiencing.

I use the get() call for one of the queries and snapshotChanges() for the other.

Sounds like another good workaround without adding additional index :)

@ColinBrilliantLabs

@ColinBrilliantLabs that would cause duplicate read costs. You can use get() as a promise based return of the documents

you cant. tradeoffs sir :)

@ColinBrilliantLabs

@ColinBrilliantLabs a limitation of firestore is that you can only use a where filter on a single field in a query. If you don't need the where filter for another purpose, you can combine it with an updatedAt field to only return the results that have changed since your last query. For example,

const lastQuery = new Date();
const users = await this.userService.getUsersCollection().ref.get();
// then read the snapshot results to see what has been added, updated, or removed and update the users value as appropriate
this.userService.getUsersCollection({after: lastQuery}).snapshotChanges();

Because firestore only charges you for documents returned, this method wouldn't charge you twice. Obviously it's a lot of work for this use case, but I also use this general strategy to ensure that a user navigating between pages doesn't reload documents which have already been loaded and cached. See firebase/firebase-js-sdk#1551 for some more info.

Is it possible to give an update on this?
the:
filter { snap -> !snap.metadata.fromCache } on snapshotChanges()
would be great.

@LanderBeeuwsaert

@LanderBeeuwsaert I believe the AngularFire team is preoccupied by a number of releases surrounding Google I/O as well as, I imagine, the impending Angular 8 release. I wouldn't expect anything to be done with this PR for several months or longer.

@thefliik yeah, I've seen the 5.2 beta tickets. Cool stuff coming. Too bad this isn't in it. But yeah, they probably have their hands more than full.

@hiepxanh

Currently have the same issue, and using @hiepxanh's work around no longes makes it fire when there is a change to the content it is subscribed to which is what i need.

I dont undertstand what you expect? if you listen it realtime, it will alway return cache data first, then it will emit the latest data right away, that is what it mean to be, try to init cache data to make it feeling really fast. I think it's working correct, what behavior do you expect?

@hiepxanh

@hiepxanh in my case for example, I have a list of 30 tickets, when you click one, you open the details of that ticket. Then we you go back to the list of 30, first 1 appears, and then the 30. That is not the expected behavior.

I am not sure about the best solution so I think the easiest way is to expose information about if the record is served from the cache or not and then let the developer take control.

@leandroz

@leandroz can you repeat it on stackbliz example repository, I will debug it

@hiepxanh

@hiepxanh not sure why you feel an example repository is necessary. The behavior @leandroz describes (which this issue is specifically addressing) is not in question. It was confirmed as "working as intended" by @jamesdaniels in a comment above. James also proposed a solution to address this issue in that comment, which (I think) would close this issue from a technical perspective (separately, these is also the issue of "surprise," which could be addressed by some documentation improvements).

I think I have a similar problem. Again because it is how firestore caches in angular it was a tough one to find. Basically the problem is : If I query on the same firestore collection twice in the same angular library then the second query is based on the subset of documents returned in the first query.

In my case I am loading a subset of documents (Checklists of type template) during onInit where the document has a property that isTemplate=true. Latter in the component I want to extract another set of documents from the same collection, but it is always selecting from the "isTemplate=True" subcollection done in the OnInit. See attached video for example.

As a work around I am going to move getting the list of templates to another component but I shouldn't need to do this. It would be nice to be able to tell AngularFirestore library not to cache the results of the first query, in this case.

My Video3.zip

So I came up with a solution, but it's pretty hacky. Basically, I have a value change that fires off every time the db is updated, and then in that value change I have a get() for the exact same documents. This way the get it called every time the db updates, and doesn't get the cached results.

For those that had trouble figuring out how to use the get method:
this.userService.getUsersCollection().valueChanges().subscribe(() =>{
this.userService.getUsersCollection().ref.get().then(usersArray=>{
             usersArray.forEach(user => {
                 
                 this.users.push(user.data());

I'm trying this approach because we wanted to show lives from youtube in our Ionic app and we were having a problem where after opening the app (through a notification) the user could see a past live from another day that was cached (i know that firestore sends the server data after the cached data, but this don't happen sometimes, not sure why).

I was worried because .get usually takes a lot of time, for instance, we have a news page where we query 10 news from a 5000 news collection and sometimes it takes 5sec over .get. But doing .get() after a .snapshotChanges() brings the results almost in real time (50ms for every request).

Metadata is now included in snapshots in 6.1, so you can filter out the cache if you want.

Much appreciated!!!!!!

@thefliik, may I ask if this solution helped in your case?

@jamesdaniels

@jamesdaniels May I know how to use the metadata to filter out the cache as you described?

@choopage I don't think that provided solution completely solves the problem that was described originally, but I so far came up with the following solution to skip the first cached push:

 public collectionSnapshotChanges(query?: QueryFn): Observable<Array<T>> {
        return this.getPrivateCollectionRef(query).pipe(
            mergeMap(ref => ref.snapshotChanges()),
            filter((actions, index) => index > 0 || !actions.every(({payload: {doc: {metadata}}}) => metadata.fromCache)),
            map(actions => actions.map(({payload: {doc}}) => ({...doc.data(), id: doc.id}))),
        );
    }

@DogAndDoll

@DogAndDoll Correct this doesn't solve the "problem" as this is Firestore behaving as designed, this is however a much better work around. Thanks for typing up the example.

@DogAndDoll

@DogAndDoll I loved your solution, and it worked at the beginning, but now I'm not receiving any data on an observable.

What I thought is that, if during the first call I have N cached items returned and the second time D items where N is different from D filter is called twice.
However, if the cached data corresponds to the remote data the library executes the filter only once, hence not being passed to the component.
Is this possible? It's really annoying having such an unpredictable behaviour

@DogAndDoll

sorry but this issue really does crown angular fire as a hunk a junk - you cant even turn off the cache behaviour :( you instead have to use @DogAndDoll's solution, switching every use of valueChanges to snapshotChanges - unbelievable

Navigation Menu

Repository navigation

unexpected: AngularFirestore#collection() returns cached value on initial subscription #2012

Version info

How to reproduce these conditions

Debug output

Expected behavior

Actual behavior

example:

10 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

unexpected: AngularFirestore#collection() returns cached value on initial subscription #2012

Description

Version info

How to reproduce these conditions

Debug output

Expected behavior

Actual behavior

Activity

diginikkari commented on Feb 28, 2019

example:

jamesdaniels commented on Mar 21, 2019

jek-bao-choo commented on Mar 22, 2019

tmk1991 commented on Mar 22, 2019

jek-bao-choo commented on Mar 22, 2019

jek-bao-choo commented on Mar 22, 2019

tmk1991 commented on Mar 22, 2019

jorroll commented on Mar 22, 2019

jek-bao-choo commented on Mar 22, 2019

tmk1991 commented on Apr 25, 2019

tmk1991 commented on Apr 25, 2019

jorroll commented on Apr 26, 2019

LanderBeeuwsaert commented on May 2, 2019

jorroll commented on May 10, 2019

LanderBeeuwsaert commented on May 10, 2019

10 remaining items

hiepxanh commented on Jun 24, 2020

leandroz commented on Jun 24, 2020

hiepxanh commented on Jun 25, 2020

jorroll commented on Jun 25, 2020

somervda commented on Jun 27, 2020

sagits commented on Jul 15, 2020

jamesdaniels commented on Nov 18, 2020

jorroll commented on Nov 18, 2020

DogAndDoll commented on Nov 18, 2020

jek-bao-choo commented on Nov 18, 2020

DogAndDoll commented on Nov 19, 2020

jamesdaniels commented on Nov 19, 2020

matteobucci commented on Jun 7, 2021

danday74 commented on May 26, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions