# Production deployment plan This is an operations plan for the current repository, reviewed on 2026-09-28. It is not a record that production has been deployed. The sysadmin should fill in the server-specific values and complete the gates in order. Use a dedicated HTTPS hostname at the web root if possible; the existing development installation uses a subpath, and its `APP_URL` must not be copied to production. ## 1. What is being deployed | Part | Current implementation | Production consequence | | --- | --- | --- | | Web app | PHP 8.3+ requirement, developed on PHP 8.5; Laravel 13.33, Blade, Livewire 3.8 and Volt 1.11 | Web server must run the chosen PHP version through PHP-FPM and point at `public/` | | Auth | Breeze routes for registration, login, password reset and email verification | Working outbound mail is needed for resets and verification | | Data | MySQL via Eloquent migrations; database sessions, cache and queue | MySQL must be available to both PHP-FPM and CLI; database backups include user and job data | | Background work | Jobs in `app/Jobs`; `routes/console.php` currently schedules one short-lived database queue worker every minute | This is not sized for a 100-video burst. The production target in section 8 uses supervised video and routine-job workers; cron remains for scheduled tasks | | AI and ad platforms | `laravel/ai`, Google Ads PHP client, Gemini, OpenAI, Google/Meta/LinkedIn integrations, optional Vertex AI and other providers | Credentials, account permissions and outbound HTTPS are feature-specific prerequisites | | Browser automation | `package.json` has Puppeteer 24, stealth plugin and scripts for catalog discovery, Meta Ad Library and brand extraction | Node, npm, browser binaries and Chrome system libraries are required for these features | | Frontend | Blade plus checked-in `public/assets`; Bootstrap and icons from jsDelivr | No Vite bundle or `npm run build`; browser users need CDN access unless assets are self-hosted later | `README.md` currently says Node/npm are unnecessary. That statement is stale: `config/creative.php` selects the Puppeteer Meta Ad Library provider by default, and `WebsiteBrandSignalExtractor` directly invokes a Node script. `npm ci` is a runtime dependency installation here, not a frontend build. The repository also has a local `setup.sh` aimed at development, with development defaults and permission changes. Do **not** run it in production. Use the explicit steps below; never copy the local `.env`, `.env_back`, `vendor/`, `node_modules/` or local database to production. ## 2. Decisions and prerequisites for the sysadmin Record these before installation: production hostname, Linux distribution, web server (Nginx example below), PHP-FPM socket/service and Unix user, deploy user, project path, MySQL host/database/user, TLS certificate method, SMTP provider, backup location and retention, monitoring destination, and which external integrations will be enabled at launch. Use one consistent deploy identity for Composer, npm, cron and file ownership. Confirm whether the initial release will permit public registration and live ad publishing. 1. Provision a Linux host sized for the site and the concurrent browser/AI workload. Give it outbound DNS and HTTPS access to Composer, npm, Puppeteer browser downloads, Google/OpenAI/Meta/LinkedIn APIs and the CDN. Permit inbound HTTPS (and HTTP only for redirect/certificate issuance). Keep MySQL private to the app host/network. 2. Install PHP-FPM plus CLI for the **same supported PHP version**. Laravel 13 requires PHP >=8.3 and its core extensions: Ctype, cURL, DOM, Fileinfo, Filter, Hash, Mbstring, OpenSSL, PCRE, PDO, Session, Tokenizer and XML. This project also needs `pdo_mysql` and `gd` (`gd` resizes Gemini-generated presenter portraits); check `php -m` for `intl`, `zip`, `bcmath`, `pcntl` and `posix` as package/feature needs dictate. Verify the selected PHP binary with `php -v` and `php -m`; on the existing development box the command is `php8.5`. 3. Install a current Composer 2 release, Git, MySQL server/client or access to managed MySQL, and a supported Node LTS with npm. Check the installed Puppeteer 24 engine requirement in `package-lock.json`/`npm view` before pinning Node. Install Chrome runtime libraries and fonts for the server distribution. If the catalog's Firefox path is enabled, install its Puppeteer-managed Firefox and native dependencies too. 4. Provision MySQL with UTF-8/utf8mb4, a dedicated least-privilege application account and a separate migration-capable account if policy requires it. The app needs read/write access to its tables; migrations need schema changes. Do not import `schema.sql` into a fresh Laravel database and then run all migrations unless the schema ownership has been reviewed; the migrations are the source of truth. 5. Arrange TLS, DNS, application/service monitoring, encrypted off-host backups and a way to restore them. Provide SMTP credentials and a verified sender domain; the default `MAIL_MAILER=log` does not deliver password reset or verification mail. ## 3. Resolve these before live traffic These are observed repository issues, not server settings to guess around: 1. **Queue visibility versus job duration.** `config/queue.php` defaults `DB_QUEUE_RETRY_AFTER` to 2160 seconds, while several jobs declare `$timeout = 2000` seconds and video declares 1500. Set `DB_QUEUE_RETRY_AFTER=2160` or a larger reviewed margin so a live job does not become available for duplicate processing. Verify the effective cached value after deployment. Keep the worker/job timeout lower than `retry_after`. 2. **Queue ownership and overlap.** The current scheduled worker has a `withoutOverlapping(10)` lock even though jobs can run for more than 30 minutes. For the 100-video production target, remove this scheduled queue worker and run the separate, supervised workers in section 8. Retain the scheduler cron for `platforms:refresh-tokens` and future scheduled commands. Never run both worker designs against the same queues. If the single-worker design is used temporarily on staging, extend its lock beyond the longest job and verify no second worker starts. 3. **File ownership.** `config/filesystems.php` currently creates public assets with world-writable permissions, and notes a split between web and cron users. Have the sysadmin assign PHP-FPM and cron to one application user, or use a shared group with setgid/ACLs. Confirm every user can write `storage/` and `bootstrap/cache/`; do not use `chmod -R 777` as a production fix. Review the application's public-disk permissions in the same change. 4. **Secrets in development tooling.** `setup.sh` contains development fallback credentials. Treat any real credentials there as exposed, rotate them, and provision fresh production credentials through the server's secret process. Keep `.env` and service-account JSON outside web access, mode 600, and out of Git/backups that are not encrypted. 5. **Long synchronous requests.** Some Livewire actions generate images or research sites in the HTTP request; brand extraction allows a 500-second Puppeteer subprocess. The video wizard also resolves a brief and title before dispatching the render job. Test the complete 100-user submission path, not just the queue. Move slow preparation out of PHP-FPM if acknowledgements or other site traffic stall. 6. **Video provider verification.** The wizard offers 8, 15 and 30 seconds. The factory routes 8 seconds to Gemini Veo and 15/30 seconds to Vertex Veo, rejecting the retired 12-second Sora path. The two Google provider paths have separate credentials and quota requirements; verify each in staging. Avatar thumbnails are not supplied as reference images, and OpenAI voice previews are not the final Veo soundtrack; assess presenter resemblance and speech quality with real staging renders. OpenAI [shut down the Videos API and `sora-2` on 2026-09-24](https://developers.openai.com/api/docs/deprecations), with no listed replacement. ## 4. Build the production host The following is a template. Replace ``, ``, ``, ``, `` and group names with values the sysadmin verified. Use a release directory or Git checkout outside the web root; only `public/` is web-served. ```bash # Run as the deploy user in after checking out a tagged commit. -v -m composer --version node --version npm --version composer install --no-dev --prefer-dist --no-interaction --optimize-autoloader npm ci --omit=dev ./node_modules/.bin/puppeteer browsers install chrome # Add Firefox only if the configured scraping fallback needs it: ./node_modules/.bin/puppeteer browsers install firefox ``` On the current development machine, Composer must be invoked as `php8.5 /usr/bin/composer`; a new production host may have a different safe command. The committed `.npmrc` sets `ignore-scripts=true`, so `npm ci` does **not** download browsers through `postinstall`; the explicit Puppeteer browser command is required. Ensure the PHP-FPM/cron identity can read the browser cache, or set and share a writable `PUPPETEER_CACHE_DIR` for installation and runtime. Test a real headless launch under that identity. The browsers and Node modules are server dependencies even though no frontend compilation runs. Create `.env` from `.env.example` in the release, then set at least: ```dotenv APP_NAME=AdMedia APP_ENV=production APP_DEBUG=false APP_URL=https:// DB_CONNECTION=mysql DB_HOST= DB_PORT=3306 DB_DATABASE= DB_USERNAME= DB_PASSWORD= SESSION_DRIVER=database CACHE_STORE=database QUEUE_CONNECTION=database DB_QUEUE_RETRY_AFTER=2160 FILESYSTEM_DISK=local CREATIVE_DISK=public MAIL_MAILER=smtp MAIL_HOST= MAIL_PORT= MAIL_USERNAME= MAIL_PASSWORD= MAIL_FROM_ADDRESS= MAIL_FROM_NAME=AdMedia SESSION_SECURE_COOKIE=true LOG_CHANNEL=stack LOG_LEVEL=warning ``` Generate `APP_KEY` **once** on the production installation with ` artisan key:generate --no-interaction`; preserve it across releases and restore it with the database. Changing it later invalidates encrypted values and sessions. Set `SESSION_DOMAIN` only if cross-subdomain sessions are needed; for a dedicated hostname, leave it empty. If a reverse proxy terminates TLS, make sure the proxy sends the original scheme and host, and verify generated links and secure cookies. `bootstrap/app.php` currently trusts all proxies, so the edge must prevent untrusted clients from supplying spoofed forwarded headers. Configure feature credentials only for features approved for launch. Main examples: `GEMINI_API_KEY` with `GEMINI_IMAGE_MODEL=gemini-3.1-flash-image` for stock presenters and `GEMINI_AUDIO_MODEL=gemini-3.8-flash-tts` for voice previews, `OPENAI_API_KEY`; `VERTEX_APP_ID` plus a private `VERTEX_AI_CREDENTIALS_PATH` JSON for Vertex video extension; Google Ads developer token/OAuth client/allowed account; Meta and LinkedIn access values; `PLATFORM_ALLOWED_AD_ACCOUNTS` and publishing guardrails; `APIFY_TOKEN` only if that provider is selected. Read `.env.example`, `config/creative.php`, `config/services.php`, `config/platforms.php` and `config/google-ads.php` for the exact keys. The default competitor-ad provider uses Puppeteer, so setting `APIFY_TOKEN` alone does not replace it. Register the production Google Ads OAuth callback URL from `php artisan route:list --name=google-ads.oauth.callback` with the provider after the hostname is fixed. Never paste tokens into tickets or logs. Web server example (adapt PHP socket, user, TLS/certificate directives and local policy): ```nginx server { listen 443 ssl; server_name ; root /public; index index.php; charset utf-8; client_max_body_size 25m; # review against actual creative upload limits location / { try_files $uri $uri/ /index.php?$query_string; } location ~ ^/index\.php(/|$) { fastcgi_pass unix:; fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name; include fastcgi_params; } location ~ /\.(?!well-known).* { deny all; } } ``` Also configure the HTTP to HTTPS redirect and a valid certificate. Test the web server configuration before reload. Do not serve the repository root or expose `.env`, `storage/app/private`, Composer files or service-account JSON. Create the Laravel public storage link and verify `/storage/...` assets work: ```bash artisan storage:link --no-interaction ``` ## 5. First release sequence 1. Take a backup/snapshot if an existing database or app is being replaced. Record the Git commit and package lockfiles. Keep production `.env`, browser cache and persistent `storage/` outside disposable releases (or copy/link them safely before switching releases). 2. Run dependency installation above. Set permissions for `storage/` and `bootstrap/cache/`, including nested directories created by cron, the browser cache and private credential files. Confirm both CLI and PHP-FPM run with the intended identity. 3. Configure `.env`, generate `APP_KEY` once, and verify DB connectivity without printing credentials. Then run ` artisan migrate:status --no-interaction` and ` artisan migrate --force --no-interaction`. Review pending migrations on every subsequent release before applying them. Seed the required video catalogs with ` artisan db:seed --class=AvatarSeeder --force --no-interaction` and ` artisan db:seed --class=VoiceSeeder --force --no-interaction`; these seeders are idempotent and ship the Gemini presenter images and voice choices. Do not run unrelated demo seeders in production. 4. Run ` artisan optimize:clear --no-interaction` and then ` artisan optimize --no-interaction` after the final environment values are present. Avoid `optimize:clear` during normal traffic: with a database cache it can clear application cache/locks. Confirm `config:show app.env`, `config:show queue.default` and `config:show queue.connections.database.retry_after` show the intended values. 5. Validate `php artisan route:list --except-vendor`, `php artisan schedule:list --no-interaction`, the web server configuration and the HTTPS `/up` health route. `/up` proves Laravel boots, not that MySQL, mail, browsers or third-party APIs work. 6. After the section 8 queue migration is deployed, install the scheduler cron **once** under the app user: ```cron * * * * * cd && artisan schedule:run --no-interaction >> /storage/logs/scheduler.log 2>&1 ``` Use an absolute PHP path, keep the log writable and rotated, and confirm `schedule:list` contains the daily token refresh but **no** scheduled `queue:work`. Install and start the supervised queue workers separately as described in section 8. Cron must not start a second set of workers. 7. Open traffic after the acceptance checks below. Keep a release/rollback marker and sysadmin contact available during the first live hours. ## 6. Acceptance checks - HTTPS `/up` returns 200; `/` and login render without mixed content, missing static assets, or 500s. Confirm assets from `public/assets` and `public/storage` load. Inspect recent `storage/logs/laravel.log`, PHP-FPM and web server error logs. - Register or sign in with an approved test account; exercise email verification/password reset and receive the message in a real inbox. Check sessions persist across page loads and that cookies are Secure. - Exercise one harmless queued generation in staging, then in production with an approved account and API budget. Observe the job in `jobs`/`failed_jobs`, its completion in the UI and files on the intended disk. Check that a job longer than 90 seconds does not reappear or execute twice. Run the section 8 burst and worker-crash checks before claiming 100-user capacity. - Test one Puppeteer Meta Ad Library search and one website brand/catalog extraction on staging under the same Unix identity used by PHP-FPM/cron. A Node version check alone does not prove Chrome launches. - Test each enabled provider using its own low-cost/sandbox action. Do not test live ad publish against a spend-capable account without a separate business approval. Verify OAuth redirect URI and account allowlists. - Verify off-host backup and a **restore drill** for MySQL, `storage/app` (public and private), `.env`/`APP_KEY` and private service-account files. Restrict backup access and retention. ## 7. Repeat deployments, monitoring and rollback For every release: review migrations and credentials, back up data, deploy a tagged commit, install from `composer.lock` and `package-lock.json`, run migrations, refresh caches, and use ` artisan queue:restart --no-interaction` after supervised workers are installed. Allow active renders to finish before removing an old release; Supervisor must restart workers against the new release. Monitor failed jobs (` artisan queue:failed --no-interaction`), queue depth/oldest job age, cron freshness, disk space, browser crashes, PHP-FPM errors, outbound mail and HTTPS uptime. Arrange log rotation and clean old browser/output files only after checking retention needs. Rollback code by restoring the previous release and its matching `.env`/dependencies, then refreshing caches. Laravel migrations are not automatically reversible in a safe production rollback; for destructive schema changes use a tested forward fix or a database restore from the pre-release backup. Restore `storage/` and the original `APP_KEY` together with the database. If jobs are failing, pause new video intake and stop the affected Supervisor worker program after active renders drain; leave the scheduler cron running for token refresh. Inspect `failed_jobs`, vendor operation status and application logs before resuming. Retry only jobs whose external effects are known to be safe. ## 8. Capacity rollout for 100 simultaneous video requests **Planning assumption:** 100 users each request one video; one render occupies one PHP queue worker while it calls and polls the external provider for 3–5 minutes. A single wizard request can request up to three videos, so 100 users can produce 300 jobs. These are capacity estimates, not a provider SLA. | Concurrent video workers | 100 one-video jobs, ideal render time | 300 jobs, ideal render time | | --- | --- | --- | | 1 (current scheduled design) | 5 h–8 h 20 min, plus up to one cron interval between long jobs | 15–25 h, plus cron gaps | | 5 | 60–100 min | 3–5 h | | 10 (initial design target) | 30–50 min | 90–150 min | The table assumes steady 3–5-minute renders and no provider throttling, failures, upload time, or synchronous preparation. Calculate any other target as `ceil(jobs / effective_workers) × measured average or worst-case render time`. Set an agreed completion target and budget with product/operations before choosing the final worker count. Ten workers are a **candidate**, not a safe setting until measured and allowed by both providers. ### Phase A: unblock the video product and establish limits 1. Confirm the retired 12-second Sora option stays disabled after deployment, then verify 8-second Gemini Veo and 15/30-second Vertex Veo generation independently, including service-account access and extension behavior. Check each account's current video concurrency, request and spend limits; [Google documents project-specific limits in AI Studio](https://ai.google.dev/gemini-api/docs/rate-limits). Obtain an approved cost per video and a spend ceiling for the 100-video test and launch. [OpenAI's deprecation notice](https://developers.openai.com/api/docs/deprecations) lists no Sora replacement. 2. Measure PHP-FPM latency and memory during the **whole** wizard flow, including brief resolution, script/title generation and render dispatch. Make final submission persist a queued creative and acknowledge it promptly. If synchronous AI calls cause saturation, move preparation into its own queued step and show `preparing`, `queued`, `processing`, `completed` and `failed` states. The rest of the site must remain responsive under 100 submissions. 3. Enforce per-user active/pending video limits and a global pending-job/spend limit before billable work. Show queued status and an approximate wait estimate based on backlog and active workers. Prevent repeated clicks from creating duplicate billable creatives. Define cancellation before vendor submission; once a vendor accepts a render, cancellation and retry require vendor-specific handling. ### Phase B: separate queues and run supervised workers 1. Keep `QUEUE_CONNECTION=database` for the first measured deployment; no new dependency is required for queue separation. Dispatch `RenderVideoCreativeJob` to a `videos` queue. Leave routine image/catalog/token jobs on `default`. Remove only the scheduled `queue:work` entry from `routes/console.php`; keep the daily `platforms:refresh-tokens` schedule. Ensure no old cron worker remains during cutover. Existing `default` jobs must drain or be deliberately migrated before the new video-only consumers start. 2. Set `DB_QUEUE_RETRY_AFTER=2160` (or a larger value based on the actual longest job). The current maximum declared job timeout is 2000 seconds; video is 1500. Keep `retry_after` strictly above all worker/job timeouts, verify the cached value, and ensure PHP CLI has `pcntl` and `posix`. Set Supervisor `stopwaitsecs` above the longest job so deploy/restart can finish it. Do not automatically retry an ambiguous vendor render: the video job has `$tries=1` because submitting it twice can incur two charges. Persist the vendor operation ID and reconcile unknown outcomes before adding automatic retries. 3. Ask the sysadmin to install Supervisor (or an equivalent process manager) and create **two** programs under the app Unix user. The following template assumes measured host and provider capacity support 10 concurrent video renders; begin with `numprocs=2`, test, then increase to 5 and finally 10. Use absolute paths and log rotation: ```ini [program:arb-video] process_name=%(program_name)s_%(process_num)02d directory= command= /artisan queue:work database --queue=videos --sleep=3 --timeout=1500 --max-time=3600 user= numprocs=10 autostart=true autorestart=true stopasgroup=true killasgroup=true stopwaitsecs=2160 redirect_stderr=true stdout_logfile=/storage/logs/video-worker.log [program:arb-default] process_name=%(program_name)s_%(process_num)02d directory= command= /artisan queue:work database --queue=default --sleep=3 --timeout=2000 --max-time=3600 user= numprocs=2 autostart=true autorestart=true stopasgroup=true killasgroup=true stopwaitsecs=2160 redirect_stderr=true stdout_logfile=/storage/logs/default-worker.log ``` `--max-time` recycles a worker **between** jobs; it does not terminate an in-flight video. Keep the cron line from section 5 for scheduled commands only. On deploy run ` artisan queue:restart --no-interaction`, then check Supervisor has restored the intended worker count. Laravel documents [Supervisor setup, worker restarts, and timeout rules](https://laravel.com/framework/docs/queues). 4. Size PHP-FPM separately from queue workers. Measure memory per PHP-FPM child and queue process at peak, then provision RAM, CPU, MySQL connections and disk I/O for that total plus headroom. All processes that write `storage/` should share a safe Unix owner/group. Store generated video files on persistent storage; size free space and backups for the expected daily volume. Moving to Redis/Horizon is a later option only if MySQL queue latency or contention warrants the added service/dependency. ### Phase C: prove the target and operate it 1. In staging, use fake video providers for a 100-submission load test so the test does not spend money. Measure acknowledgment latency, FPM saturation, duplicate submissions, queue depth, oldest-job age, worker crashes, MySQL load and completion distribution. Test a simulated 3–5-minute provider delay with 2, 5 and 10 workers. Also test a killed worker, provider 429/503, storage failure, deploy during a render, and loss/recovery of a vendor operation ID. Run a small **approved** live parallel test for real provider capacity and output quality. 2. Launch at the measured safe worker count; ramp upward only when vendor limits, server headroom and spend budget permit. Alert on missing workers, rising oldest-job age, failed jobs, provider throttling, failed token refresh, insufficient disk, and unexpectedly high API spend. Provide an operator procedure for pausing intake, letting active jobs drain, inspecting failures, reconciling vendor operations, and resuming. Avoid indiscriminate `queue:retry all` on billable video jobs. 3. If the 100-job burst still exceeds the agreed completion target, add workers only within proven quotas or split vendor submission and status polling into separate jobs so long provider waits do not occupy PHP workers. Re-measure after any such change. Keep one queue owner for each queue at every stage of rollout and rollback. ## References - [Laravel 13 deployment](https://laravel.com/framework/docs/deployment) - [Laravel 13 queues](https://laravel.com/framework/docs/queues) - [Laravel 13 task scheduling](https://laravel.com/framework/docs/scheduling) - [Puppeteer system requirements](https://pptr.dev/guides/system-requirements) and [installation](https://pptr.dev/guides/installation)