Repository navigation
feature: faster article list endpoints (no DISTINCT joins, list indexes, batched GraphQL authors) - #1256
feature: faster article list endpoints (no DISTINCT joins, list indexes, batched GraphQL authors)#1256matthewguerra-cog wants to merge 1 commit into
Conversation
…ins) and batched GraphQL author profiles
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
There was a problem hiding this comment.
✅ Devin Review: No Issues Found
Devin Review analyzed this PR and found no bugs or issues to report.
Custom reviewers
|
Migration Engineer review — SQL/schema safety vs #1252: no blocking issues.
Non-blocking: the Flyway |
|
Review of #1256. I found no blocking bugs. The new IN-subquery filters behave the same as the old
Gaps filled (#1257):
Not yet run: |
Summary
Article list endpoints were slow for two reasons:
DISTINCT.queryArticles/countArticle/findArticlesWithCursorjoinedarticle_tags,tags,article_favoritesanduserstwice on every call (even with no filter), then de-duplicated withDISTINCT/count(DISTINCT)and sorted in a temp B-tree. Filters are nowINsub-selects on a plainarticlesscan, so no join, noDISTINCT:Article.authorcalledProfileQueryService.findByUsernameper article (2 statements each → 46 statements for 20 articles). It now goes through a DGSMappedBatchLoader(ProfileDataLoader) backed byProfileQueryService.findByUsernames(oneusers ... in (...)+ onefollowingAuthorsquery).Plus
V2__add_article_list_indexes.sql:article_tags(tag_id),article_favorites(user_id, article_id),follows(user_id, follow_id)(allcreate index if not exists, additive only). I deliberately did not indexarticles(created_at)orarticle_tags(article_id): with sqlite-jdbc 3.36 either one changes the hydration plan infindArticles/findArticlesOfAuthors, which have noORDER BYfor tags, sotagListorder changed. Without them, the responses are byte-for-byte the same as before.DefaultJwtServiceTesthas a one-line formatting fix only becausespotlessJavaCheckfails onmasterwithout it.Before / after
Seeded SQLite: 50 users, 2,000 articles, 30 tags (4,054 article-tag rows), 5,000 favorites, 500 follows (sample values,
random.seed(42)). Authenticated as a user who follows 10 authors. 30 warm-up requests, then 200 sequential requests per endpoint; both jars ran on the same box with identical DB copies. SQL count = MyBatis==> Preparinglines per request.GET /articlesGET /articles?tag=tag7GET /articles/feedarticles(first: 20)REST statement counts stay at 7 because
fillExtraInfoalready batches; the REST win comes from cheaper statements. Response parity: 26 REST and GraphQL requests (list, tag, author, favorited, combined filters, offsets, feed, cursor first/last, anonymous and authenticated) return identical bodies before and after../gradlew spotlessJavaCheck buildpasses on JDK 11; a newProfileQueryServiceTestcovers the batch lookup, including following state and missing usernames.Overlaps with open PRs
MappedBatchLoader/@DgsDataLoaderare unchanged in DGS 5.6. Migration Engineer verified Upgrade Spring Boot to 2.7.18 (DGS 5.6.2, MyBatis starter 2.3.2, sqlite-jdbc 3.49.1.0) #1252 + feature: faster article list endpoints (no DISTINCT joins, list indexes, batched GraphQL authors) #1256 merge cleanly, build green, and responses stay byte-identical on sqlite 3.49 (comment).V2collision: Add BM25-ranked full-text article search endpoint #1121, feature: golf one-and-done data model, migration and seed data #1233, bug: hash updated passwords with BCrypt and re-hash stored plaintext (sfind-4d730a24) #1174, Add password reset endpoints (COG-987) #1020 and Add CRM module for LPs, activities and pipeline stages #869 also add aV2__migration. Whichever lands second needs to be renumbered.ArticleReadService.xml; feature: migrate backend from Spring Boot + MyBatis to Django + DRF #1150 rewrites the backend entirely.Devin-Org: engineering
Implemented in session Speed up RealWorld API article list endpoints
Review with Devin · Open in Devin Desktop