Skip to content

Route C: develop@4a3b74550 with Symfony 7.4, PHP 8.4, ORM 3 and Manticore 29 (#519) - #582

Draft
Neophytis wants to merge 270 commits into
mainfrom
neophytis/519-route-c-4a3b
Draft

Neophytis wants to merge 270 commits into
mainfrom
neophytis/519-route-c-4a3b

Conversation

@Neophytis

@Neophytis Neophytis commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Target branch: main

Part of #519 (route C: upgrade the framework first, redesign the schema later).

What:

Why: prod runs a framework line that is no longer developed; this brings it onto develop's code without changing the database.

Checked:

  • CI green on every check at 8f7c4e818 (commits re-signed, same files as sha-4e997f2).
  • Stage, prod data copy: full e2e suite green on all 4 browser jobs, last on 2026-10-15 (run).
  • Switch and rollback between main and this branch rehearsed on stage with plain deploys; the deploy swaps the Manticore data per version (sysadmins-infra #740).

Rollback: deploy 3.8.5 again; the deploy puts Manticore 6.3.8's data back; the database needs nothing (no migrations).

Before merging:

  • Name split test merged (sysadmins-infra #749, green on route C and on main): one name field, last word becomes the last name (c26d472).
  • Manticore 29: multi-word place names search as on prod (2026-10-16, stage vs prod, same top 3: New York, Rio de Janeiro, Den Haag, Saint-Denis, Frankfurt am Main, Ho Chi Minh, Thai กรุงเทพ / กรุงเทพมหานคร).
  • Questions for review answered (below).

Questions for review

  1. Is 4a3b74550 the right base, or should this start from a later develop commit and keep the old members table some other way?
  2. Sign-up: keep main's flow until the schema step (as Privacy: member email addresses in /signup/resend URLs reach Plausible and logs (low priority) #541 suggests, develop's sign-up comes with the new tables)?
  3. Profile comments: main sorts them (new experiences first), develop shows the last 10. Which one?
  4. Prod's migration history is empty, so the first migrating deploy will fail #560: remove the 2020 migrations, as suggested. The functions and the forum_trads view from Version20200307133359 then need another way into fresh databases (devcontainer, CI), for example a SQL file next to languages.sql.
What changes, in detail

develop at 4a3b74550 (2025-10-20) is the last develop commit that still runs on prod's database. One commit later (7ecf67c4c, "Introduce new member entity") develop moves the login and about 160 files to the new member table. At 4a3b74550 it already has Symfony 7.4, PHP 8.4 and Doctrine ORM 3, and its CI was green.

On top of it, everything needed to run it the way prod runs:

Change Source
Dockerfile (PHP 8.4.26), docker/, docker-compose.yml, .trivyignore, image workflows main
Symfony 7.4.x-dev to 7.4.20 releases (SYMFONY_REQUIRE=7.4.*), Twig, Guzzle patch releases composer update, audit clean
Intervention Image v4 (300d2e2e2, #535) main
circle-flags and @tomickigrzegorz/autocomplete from npm (the GitHub sources need git in the build; the autocomplete commit no longer exists) develop
Doctrine reads DB_* instead of a DATABASE_URL built in .env; the session handler reuses the Doctrine connection main
translations:add:missing: bind the three parameters develop
ParamConverter docblocks to #[MapEntity] (5 controllers) develop
{username:member} on 23 routes (controller_resolver.auto_mapping is false) develop
DBAL 4 parameter names without the leading : in the conversation adapters new; develop has the same bug
error500.html.twig trans({username: ...}) main and develop
public/images/logo_index_top.png main
profile.js null check for the language switch new, minimal
Branch builds no longer move the rolling stage image tags new
Test history

First stage test (prod data copy, 2026-10-09)

  • Image sha-cbf2c40: deploy and stage smoke tests green, Trivy 0 findings (OS, npm, Composer), no CRITICAL in the PHP log after deploy.
  • Browser check of 21 pages, logged out and logged in, compared with prod: same status codes and assets on every page, comment order identical. Also green: notes, relations, preferences, conversations with a member, messages, requests, invitations, forum posts by member, avatars (resized through Intervention v4), group members, gallery.
  • Same as prod (not regressions): JS errors on /faq and /feedback, /locale/fr 404, visitors page redirect.
  • Visible difference: the header. main has the new header, logos and home page (2026-04-24, 2026-05-12); this branch has develop's older header. That is part of the list below.

Later runs

main's commits since the branch point (125, ported or left out)

These were the porting list when this PR was opened (2026-10-12). All are on this branch now, except #590 and #607 (left to their author) and #603/#606 (reverted, #609). The header and home page redesign is ported.

Docker, CI and deploy (28)

Security and privacy (7)

  • 23aae920e 2024-04-27 Use session invalidation to force user logout to avoid re-activation of just retired member.
  • 58ceac4ff 2026-08-15 fix(proxy): forward X-Forwarded-Proto so Symfony generates https:// URLs
  • 5fdbc73ad 2026-08-27 fix(proxy): declare trusted_proxies/headers in framework.yaml; cookie_samesite lax
  • 0633769fb 2026-08-27 fix(proxy): trusted_headers must be YAML list in Symfony 5.4, not env string
  • aa9ed3ee2 2026-08-27 chore: remove api-platform/core (unused, accumulating CVEs)
  • 777a5d0a0 2026-09-02 Migrate cron job to symfony command.
  • 3eb41314c 2026-09-27 fix(session): keep members logged in for 7 days, remember-me samesite lax (fix(session): keep members logged in for 7 days, remember-me samesite lax #516)

Mail (9)

Frontend, layout and UX (10)

  • a44166832 2026-04-24 Add new logos everywhere. Update navigation and footer. New home page.
  • fcf91311f 2026-04-25 Fix menu for legacy pages like signup.
  • c43ed7902 2026-04-25 Fix menu for pages that work without login.
  • 46fae7b9e 2026-04-25 Fix menu for policy pages.
  • 4c1cf4337 2026-04-26 Fix menu overflow on legacy pages.
  • 6c8e69194 2026-05-06 Use new logo on error pages. Update yarn.lock for browserslist.
  • 002be5378 2026-05-12 login access from header + redesign login page
  • e7286e054 2026-05-31 Give full logo space to breath. Fix wordmark becoming too small compared to icon.
  • 0ad55a778 2026-06-10 Make sure the member menu is right aligned.
  • 5a56b5350 2026-06-13 Update web preview images. Fix old logo in base-lite template.

Bug fixes (29)

  • 21b74aa3d 2022-06-16 Fix not set entry in membersphots table.
  • b87a56540 2024-04-27 Fix problem with user authentication and only report blank errors for form fields.
  • 18d1f2ff1 2024-04-27 Fix problem with user authentication and only report blank errors for form fields.
  • a38405892 2026-04-26 Fix formatting of language dropdown.
  • 79fe269b9 2026-05-06 Fixing server errors due to wrong trans usage. Allow Ukrainian as translation language.
  • 0e0b7cbc0 2026-06-23 Fix redirect problems.
  • 3d6e26e7e 2026-08-15 fix(fpm): increase PHP-FPM worker pool from 5 to 25
  • 8ea7efaa0 2026-08-27 fix(manticore): fix geonames column name, forum query, and add negotiation dep
  • 9abfd8eab 2026-08-27 fix(nginx): set HTTPS fastcgi param so legacy PHP detects HTTPS
  • 9eb503041 2026-08-27 fix(api-platform): remove leftover bundle registration and annotations
  • 0109bda24 2026-08-27 fix: apply Shevek's review - bump APCu to 5.1.28, drop Phinx migration dead code
  • d1836d965 2026-08-30 Fix forum reply (MariaDB in safe mode; not default value for authorid)
  • 1a3d001ea 2026-09-02 Fix broken range slider on signup and edit my profile page.
  • 0491b2bdc 2026-09-05 fix(forum): use configured Manticore host/port for search
  • 045626d10 2026-09-06 fix(manticore): revert geoname_id → geonameid in SQL queries
  • 4868d2a9a 2026-09-08 fix(php): set date.timezone to Europe/Berlin to match Lion and DB
  • cc2281edd 2026-09-16 fix(translation): handle missing DB schema gracefully during cache:clear
  • 67f1b71d7 2026-09-19 Fix manticore index generation.
  • d6363ae9d 2026-09-19 fix(manticore,nginx): fix geonames rebuild and font caching (fix(manticore,nginx): fix geonames rebuild and font caching #506)
  • b25faacd2 2026-09-19 Fix display glitch on search location input.
  • 6cfaac344 2026-09-21 Fix/feat: Always show trips even if legs' location is missing in database.
  • 4a1ac75c3 2026-09-22 Fix: Newly added translations were not imported.
  • f66cc1ef5 2026-10-01 fix(upload): raise nginx and PHP upload limits to match Lion (8M) (fix(upload): raise nginx and PHP upload limits to match Lion (8M) #528)
  • e18ea1fa2 2026-10-02 fix(public): serve the lion-only verification files; https for dev seed downloads (fix(public): serve the lion-only verification files; https for dev seed downloads #537)
  • 8fad344d0 2026-10-04 Fix ValueError for translations containing %.
  • 2f94d37af 2026-10-05 fix(logging): prod logs errors to stderr, deprecations on demand (Prod writes about 2 GB of debug logs a day inside the container #524) (fix(logging): prod logs errors to stderr, deprecations on demand (#524) #558)
  • 5368d8346 2026-10-08 fix(seo): canonical tags, apex sitemap and valid robots.txt (sysadmins-infra#654) (fix(seo): canonical tags, apex sitemap and valid robots.txt (sysadmins-infra#654) #561)
  • 4e2339a54 2026-10-08 fix(terms): correct BeVolunteer RNA number (W353003546) (fix(terms): correct BeVolunteer RNA number (W353003546) #563)
  • 5f5ff1b90 2026-10-09 fix(seo): /about/faq is a permanent redirect, links go to /faq (BeWelcome/sysadmins-infra#654) (fix(seo): /about/faq is a permanent redirect, links go to /faq (sysadmins-infra#654) #564)

Other (42)

  • 51f0d0aed 2023-07-10 Add flash that is shown if comment is locked.
  • 6d620c344 2023-07-11 Set status to active after successful login (even if login was using remember me).
  • ee51b7cfb 2023-07-13 Show comment update on profile page and set ADMIN_CHECK for negative comments.
  • c6c666b1f 2024-04-27 Make /deleteprofile accessible without redirect from Play store compliance.
  • ee911ab63 2026-05-06 Add blocked messages to the admin tools.
  • 007022ed7 2026-05-09 Force checkbox appearance.
  • 5860d89b3 2026-05-11 Add more constraints for activities.
  • a111d0bb6 2026-05-11 Align master with production.
  • 975963892 2026-05-13 Further align master and www.
  • 32dd5d34f 2026-05-13 Move to Noto Sans and some alignment of master and www.
  • 229b4cfde 2026-05-31 Align login forms. Fix signup date.
  • 769066748 2026-06-06 Use full size sheet only for xs and sm screens.
  • ff0b40877 2026-06-07 Use Noto Sans on /login as well.
  • 93157bfa9 2026-08-04 Use latest ACPu extension.
  • 4ce44da92 2026-08-13 Update formatting options for messages.
  • 4d0622f5d 2026-08-15 perf(nginx): enable gzip and long-lived cache for hashed assets
  • 90137e8b6 2026-08-27 perf(homepage): cache statistics queries for 1 hour
  • 112fd85c4 2026-08-27 chore: remove api-platform Serializer normalizers left behind by 3dd46fc
  • 7db45bf3e 2026-08-27 chore: remove unused @groups annotations from entities
  • ed1517520 2026-08-27 perf(trips): add bounding-box pre-filter before haversine GeoDistance
  • f2cd5dfec 2026-08-30 chore(db): pin MariaDB to 12.3 LTS (mariadb:lts tag)
  • 965102a02 2026-09-02 Rewrite to use only PHP 7.4 features.
  • e25594574 2026-09-02 Add system to highlights.
  • 92297d20b 2026-09-02 Add external contact info.
  • 5984133e3 2026-09-02 Set correct anchor (#external is referenced in tou).
  • efabe053c 2026-09-06 Remove SQL_CALC_FOUND_ROWS everywhere.
  • 5d265d2ef 2026-09-06 Some cleanup in ForumCleanupCommand.
  • 96117b2ce 2026-09-10 Removing spurious < from template.
  • fc6bc398e 2026-09-10 feat(analytics): parameterize Plausible script URL via PLAUSIBLE_SCRIPT_URL env var
  • 0249203aa 2026-09-15 feat(analytics): add Plausible custom event tracking for key user actions
  • 70b7df3ec 2026-09-19 Revert fix for display glitch (not effective).
  • 782c9ccee 2026-09-20 Update LOCALES to include Armenian
  • d28f0215f 2026-09-20 Add necessary files to support Armenian in translations directory.
  • 02cf7e57c 2026-09-20 Use object instances instead of static access for Mod_htmlpure.
  • fcd8c91bc 2026-09-21 Feat: Add znuny to the credits page. Closes Add znuny to the credits page #510.
  • 455c60064 2026-10-01 Add Ukrainian and fix display glitch in language dropdown.
  • ec4b02939 2026-10-02 Use symfony translator instead of loading rows from the database.
  • 300d2e2e2 2026-10-02 Upgrade to Intervention image library v4.x and adapt image handling code accordingly.
  • 0dd7c6906 2026-10-04 Set absolute URL on empty referrer. Return a valid redirect response in all cases.
  • af342242f 2026-10-04 Backport fix for 548 to production source.
  • fa6a63567 2026-10-05 feat(logging): write warnings to stderr in prod as they happen (Prod writes about 2 GB of debug logs a day inside the container #524 follow-up) (feat(logging): write warnings to stderr in prod as they happen (#524 follow-up) #559)
  • 43d64020d 2026-10-09 style: bring src/ back in line with php-cs-fixer (CI fails on every push to the production branch, so PRs always show red #517) (style: bring src/ back in line with php-cs-fixer (#517) #573)

thisismeonmounteverest and others added 30 commits May 7, 2024 20:32
…f applicable. Sort translations correctly. Show/hide password.
* Add check for phone number in CommentModel

* Add check for phone number in CommentController

* Add anti-spam check in RelationsController

Added functions to check for email address and phone number, and only add the comment if none are detected
…nd make forum work with strict mode for DB.
…tes only that member's avatars (#519)

setLatitude('')/setLongitude('') into ?float and setChatGoogle(<id string>) into int threw a TypeError on the first member, so nobody was anonymised. Now null, null and 0 (v3.8.5 stored 0 there). Avatars: name('<id>*') also matched other members (12 took 120_xs, 1234); now '<id>' and '<id>_<size>' only. Checked on stage with a test member and look-alike files. Audit B2.
The three empty uk .db marker files were deleted while uk stays in LOCALES, so the database translations for Ukrainian were not loaded. Restored from v3.8.5. Audit B4.
Every autocomplete request wrote the member's search text at ALERT level; prod logs it at debug, which is not recorded. Audit B5.
Port of prod's formatConversation (also develop's): zero-width characters stripped and NFKC applied to a copy before the check, plus .shop, system and support. The message itself stays as written. ConversationModelTest fails on the old code. Audit I1.
route C's lock came from develop and had 27 packages older than prod v3.8.5, among them ezyang/htmlpurifier 4.18 (prod 4.19). composer update of those 27 with dependencies; brick/math kept on prod's 0.18 line. composer audit: no advisories. Two PHPStan baseline entries the newer PHPStan no longer reports are removed. Audit I3.
DBAL 4 rejects ':types'/':memberId' as parameter names and needs an array type for IN (); the errors were swallowed and the filtered list stayed empty. Checked on stage: the adapter's count equals SQL's (81,837). LogAdapterTest fails on the old code. Audit I5.
…s again (#519)

The legacy profile editor uses the jQuery rangeslider (.rangeslider); only develop's .rangeSlider rules were left. prod's rules added next to them; the production asset build passes. Audit I7.
Credits from v3.8.5: Znuny listed again, no Matomo tracker (matomo.bewelcome.org is stopped), the HTML Purifier badge served locally. /favicon.ico restored (2,082 requests in prod's recent access logs). Audit I9.
…519)

/signup/finalize/{username} let anyone who knew a new member's username fill in their name, birth date, gender, location and accommodation. The finalize page now opens only in the session that proved ownership: right after step 1, through the confirmation link, or after logging in with the password (UserChecker throws in checkPostAuth). Others go to the login page. develop's finishSetup has the same gap. Audit B3.
…neypot (#519)

As prod's sign-up: a hidden 'sweet' field that bots fill in stops the sign-up with prod's message, and the finalize step asks one mother tongue (spoken and signed languages grouped, prod's translated texts), stored as a MotherLanguage level like prod and develop. prod's sign-up feedback to volunteers was a hidden, always empty field; nothing to restore. Audit I8.
The validator translates in the 'validators' domain; BeWelcome's keys (signup.error.username, error.name, ...) live in 'messages', so the new sign-up showed raw keys. The form theme translates a message that is still equal to its key in 'messages'. Symfony's own messages are unchanged. develop has the same theme.
develop dropped the Shout entity in 4530357 (Symfony 7.2 upgrade) while the export page, its texts and the shouts table stayed: an oversight. Restores the read-only entity (attributes, `table` quoted) and the shouts extractor as on prod v3.8.5. Checked on stage with prod's data: all 1,931 shouts load; member 71's export source returns their 45. Audit I4.
The legacy profile page showed the last 10 comment pairs unsorted; prod shows all of them, latest activity first (a new experience moves a pair up). Restores CommentSorterProfile; CommentSorter reads dates through two overridable getters (same behaviour for CommentRepository). Checked on stage: member 13993 has 238 pairs, all shown, newest first. Audit I6.
isDeniedAccess() allowed only five statuses and refused ChoiceInactive, NeedMore, ActiveHidden and Pending members (4,360 on prod, 81 of them active in the last 30 days). Back to prod's rule (v3.8.5): refuse only members whose profile cannot be browsed. Checked on stage with real members of each status. MemberLoginTest.
…members again (#519)

Carbon 3's diffInMinutes() is signed: for a past last login it is negative, so '> 5' was never true and neither the last login nor the status was ever updated (search, statistics, reminders, retention rely on it). Uses the absolute difference. getAge() casts Carbon 3's float. AuthenticationEventSubscriberTest fails on the old code.
…message again, not an error page (#519)

checkTimeElapsedOnPasswordReset() was called with null. Checked on stage: 200 with 'Couldn't find a user with that name or email address.'
They were docblock annotations, which Symfony 7 no longer reads: creating a group, editing a photo, translations, FAQ and FAQ categories, reporting a comment, community news comments and admin change-username accepted anything. Now attributes with the same options. ValidationRulesTest fails on the old code.
…#519)

The locale switch saved the short code into PreferenceLanguage, which Member::initializePreferredLanguage() and prod's data (400,000+ rows) hold as the language id; members fell back to English at the next login and in mails.
…rs again (#519)

The getters were dropped with develop's Member rewrite and the export block with them. Restores getCryptedField(), getPhoneNumbers(), getMessengers() and getWebsite() from prod (v3.8.5) on route C's fields relation, and the template block (website loop null-safe). Checked on stage with a member's three phones and two messengers.
delete_translation_multiple() got its parameters in a new order on develop; the caller still used the old one, so nothing was deleted.
#519)

Two lines used u: classes, which this Tailwind setup (prefix u-, separator :) does not generate. Prod's classes restored.
…519)

Only six languages were bundled; prod built all of them. One lazily loaded translation per interface language (39 of 42; rm, su, sw have none, as on prod), and no translations option when there is none (CKEditor 47 fails on an empty array). Drops five script tags for build/cktranslations, which no longer exists.
offerdinner, offertour, profile_picture and about_me are now offers_dinner, offers_tour, has_profile_picture and has_about_me; old URLs are mapped. SearchLegacyParametersTest.
… goals, as on prod (#519)

base-lite and error lacked the _analytics include, so the 'Signup Confirmed' event flash showed to members as a raw alert. Restores Review Written, Message Sent, Request Sent and Host Search. Audit I2.
…e error (#519)

Profile comment read more/less labels come from the page again where it has them (prod), the library default elsewhere. check.js ended with a stray 'g' instead of getPasswordScore().
Twig syntax error ('} %})') in the template, also on prod v3.8.5; found by lint:twig.
What testers and members will notice: new, changed on purpose (feedback wanted), fixed during review (same as 3.8.5 again), fixed problems of 3.8.5, removed, known issues. Format: Keep a Changelog.
@thisismeonmounteverest

Copy link
Copy Markdown
Contributor

It should already be clear from the long, long list of check-ins and the now conflicted merge that this is no longer a proof of concept but an obstacle in bringing the site to the modern stack that as said several times already runs in develop.

Instead of spending several hours on this PoC fixing develop to run would have been way more valuable.

@Neophytis

Copy link
Copy Markdown
Contributor Author

It should already be clear from the long, long list of check-ins and the now conflicted merge that this is no longer a proof of concept but an obstacle in bringing the site to the modern stack that as said several times already runs in develop.

Instead of spending several hours on this PoC fixing develop to run would have been way more valuable.

I am not sure you took the time to actually test it before adding your comment. It's further than PoC indeed and is already better than prod as it has no regression compared to prod. But indeed develop has a lot of gaps to be covered which I tried to cover in the parts I cherry picked. If develop had solved those problems effectively I would have just cherry picked them. Also I am able to evaluate when it's too much work for me no worries.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants