Add option to include the --save-cache option to the query server to more aggressively save the cache

[Sn0rlax-lgtm]

Running a query, closing, reopening, and rerunning a query on VS Code yields no performance benefits. Is VS Code (or Eclipse/Visual Studio) configurable to use an on disk cache, similar to the CodeQL CLI?

[p0]

The VS Code extension actually uses the CodeQL CLI under the hood, so it should be using a disk cache in exactly the same way. I expect at least every cached predicate to end up in the disk cache (which you can see under db-*/default/cache in a CodeQL database folder), meaning that after a VS Code restart re-running a query should only execute the final stage (and not any of the preceding stages), assuming the QL code is the same.

Are you observing a situation where this does not happen? Would you be able to provide reproduction instructions?

[Sn0rlax-lgtm]

Ah, it turns out that restarting VS Code results in a rerun of the final stage. I'd assumed that it would be cached, but apparently not?

Running cpp/ql/src/Architecture/General Class-Level Information/ClassHierachies.ql as an example, the full query takes about 100seconds on one of my code bases, followed by 0s and then 45s when Visual Studio Code is restarted.

[p0]

This is expected -- if you keep running queries in a single VS Code session, the CodeQL engine stays up, and its memory cache is able to service repeat queries (or indeed slightly edited queries) very quickly.

If you shut down the QL engine (by restarting VS Code, or switching database), that memory cache is lost, and only the disk cache is available. We deliberately avoid caching the final query stage eagerly -- one of the main use cases for the VS Code extension is interactively refining analyses, where you run the same query with slight modifications to the final stage repeatedly. If we wrote each of those intermediate results to the disk cache, we'd use an unreasonable amount of disk space. The disk cache aims to contain predicates that are likely to be useful for future query evaluations, which in practice tends to mean cached predicates (aka non-final query stages).

Aside: The engine actually has a mode where it uses the disk cache much more eagerly. We could consider exposing that mode (ideally with a user-configurable disk cache threshold) in the VS Code extension, in order to make restarts less disruptive, though some experiments on whether it would actually save time overall or in fact spend longer writing single-use data to disk would be needed.

[Sn0rlax-lgtm]

This is expected -- if you keep running queries in a single VS Code session, the CodeQL engine stays up, and its memory cache is able to service repeat queries (or indeed slightly edited queries) very quickly.

If you shut down the QL engine (by restarting VS Code, or switching database), that memory cache is lost, and only the disk cache is available. We deliberately avoid caching the final query stage eagerly -- one of the main use cases for the VS Code extension is interactively refining analyses, where you run the same query with slight modifications to the final stage repeatedly. If we wrote each of those intermediate results to the disk cache, we'd use an unreasonable amount of disk space. The disk cache aims to contain predicates that are likely to be useful for future query evaluations, which in practice tends to mean cached predicates (aka non-final query stages).

Aside: The engine actually has a mode where it uses the disk cache much more eagerly. We could consider exposing that mode (ideally with a user-configurable disk cache threshold) in the VS Code extension, in order to make restarts less disruptive, though some experiments on whether it would actually save time overall or in fact spend longer writing single-use data to disk would be needed.

Yes, the save-cache option is exactly what I have been using in the CodeQL-CLI. I don't see any harm in making it an (advanced) option, as long as it is off by default.

[aeisenberg]

Thanks for the explanation. I've changed the title and added this as a feature request.