This PR replaces the existing `skip-to` approach for rendering large diffs with paging over the changeset of a git diff, enabling a more efficient way to render the compare view for PR, compare, and commit views.
## Problem
When a user opens a compare page for a large range of commits, Forgejo calculates the entire diff (potentially multiple times) Which consumes significant CPU time on the server and produces an enormous amount of HTML for the browser to parse and display. At some point the page becomes unusable or fails to load entirely.
The previous `skip-to` approach attempted to mitigate this by allowing the user to jump to a specific file or section within the diff. However, this still required the diff process and prepare the full changeset up to that point. It did not fundamentally bound the amount of work done per request: the server could still spend excessive time calculating the Diff. As a result, `skip-to` was not a reliable solution for very large diffs.
TLDR; `--skip-to` is an output-ordering/presentation option, not a performance optimization. [git docs](https://git-scm.com/docs/diff-options#Documentation/diff-options.txt---skip-tofile)
## Solution
This change introduces paging over the changeset of a git diff. Rather than relying on `skip-to`, the diff is now split into manageable pages. Only a subset of the changed files is fetched, highlighted, and rendered at a time.
### Tests for Go changes
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [ ] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Security features
- [PR](https://codeberg.org/forgejo/forgejo/pulls/12808): <!--number 12808 --><!--line 0 --><!--description ZG9uJ3QgcmVuZGVyIGxhcmdlIGNvbXBhcmUgZGlmZg==-->don't render large compare diff<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12808
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
### Context
Following the feedback in forgejo/discussions#170 (and my ambitious attempt in forgejo/forgejo#10985), it appears that having an easy-to-use factory package would greatly help get rid of the global fixtures.
I think that the global fixtures are quite harmful (recent example: https://codeberg.org/forgejo/forgejo/pulls/9906#issuecomment-10826066):
- hard to write (contributor must know where to add them)
- hard to change (may break some unrelated tests)
- hard to review (not located near the test code)
- they require the tests to execute sequentially
### Proposed way forward
The `forgery` package (the name represents faking/crafting and sounds good with Forgejo) is meant to replace global yaml fixtures with local go factories. The forgery can currently:
- create users
- create repos
- create organisations
This allowed me to drop `CreateDeclarativeRepoWithOptions` (and deprecate `CreateDeclarativeRepo`).
I think that further changes should be delayed to other PRs (I have a local branch to create `Project`)
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [x] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
*The decision if the pull request will be shown in the release notes is up to the mergers / release team.*
The content of the `release-notes/<pull request number>.md` file will serve as the basis for the release notes. If the file does not exist, the title of the pull request will be used instead.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/11356
Reviewed-by: limiting-factor <limiting-factor@noreply.codeberg.org>
Resolves#8428
Its currently showing the diff of two branches, even if the PR-unit is disabled. But the POST to create the PR is still returning an error when the unit is disabled, so its only a change for UI.
Preview:
https://codeberg.org/attachments/610d6b81-a50f-4c43-91c2-db4c38e7b701
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/8496
Reviewed-by: 0ko <0ko@noreply.codeberg.org>
Co-authored-by: zokki <zokki.softwareschmiede@gmail.com>
Co-committed-by: zokki <zokki.softwareschmiede@gmail.com>
- This allows `CreateDeclarativeRepo` to be used by other testing
packages such as E2EE testing.
- Removes unused function in `services/webhook/sourcehut/builds_test.go`.
This adds a new test case to `TestCompareCodeExpand` to exercise the
case where we're viewing a PR's diff.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
When comparing files between the base repo and forked repo, the "blob
excerpt" link should point to the forked repo, because the commit
doesn't exist in base repo.
Co-authored-by: Giteabot <teabot@gitea.io>
(cherry picked from commit f48cc501c46a2d34eb701561f01d888d689d60d5)
Conflicts:
- templates/repo/diff/section_split.tmpl
- templates/repo/diff/section_unified.tmpl
Resolved the conflict by picking Gitea's change over ours, and
porting it.
- tests/integration/compare_test.go
Kept our test, but picked the "compare all of the relevant
links" part of the Gitea test.
When expanding code diffs, the expansion should search for more context
in the commits repo, rather than in the repo in context, because the
commit may not be available in the base repo. For example, when
previewing a pull request, the commit is not in the target repo yet -
it's in the fork.
Fixes#3746.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
More about codespell: https://github.com/codespell-project/codespell .
I personally introduced it to dozens if not hundreds of projects already and so far only positive feedback.
```
❯ grep lint-spell Makefile
@echo " - lint-spell lint spelling"
@echo " - lint-spell-fix lint spelling and fix issues"
lint: lint-frontend lint-backend lint-spell
lint-fix: lint-frontend-fix lint-backend-fix lint-spell-fix
.PHONY: lint-spell
lint-spell: lint-codespell
.PHONY: lint-spell-fix
lint-spell-fix: lint-codespell-fix
❯ git grep lint- -- .forgejo/
.forgejo/workflows/testing.yml: - run: make --always-make -j$(nproc) lint-backend checks-backend # ensure the "go-licenses" make target runs
.forgejo/workflows/testing.yml: - run: make lint-frontend
```
so how would you like me to invoke `lint-codespell` on CI? (without that would be IMHO very suboptimal and let typos sneak in)
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/3270
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>
Co-committed-by: Yaroslav Halchenko <debian@onerussian.com>
When comparing branches, only offer those branches to use as a base
where the repository allows pull requests. Those that do not allow pull
request would result in a 404, so offering them as an option would be
misleading.
Refs: https://codeberg.org/forgejo/forgejo/pulls/2194
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
(cherry picked from commit 022d0e0d71)
(cherry picked from commit 957990b36a)
(cherry picked from commit 6d2df72825)
Change all license headers to comply with REUSE specification.
Fix#16132
Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
The `go-licenses` make task introduced in #21034 is being run on make vendor
and occasionally causes an empty go-licenses file if the vendors need to
change. This should be moved to the generate task as it is a generated file.
Now because of this change we also need to split generation into two separate
steps:
1. `generate-backend`
2. `generate-frontend`
In the future it would probably be useful to make `generate-swagger` part of `generate-frontend` but it's not tolerated with our .drone.yml
Ref #21034
Signed-off-by: Andrew Thornton <art27@cantab.net>
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
