Written 2026-08-10 after a copy operation left /var/www/ehhspa unreadable to its own web server. Every fact below was measured on this box, not assumed. Where something is counter-intuitive the evidence is included, because the counter-intuitive ones are what bite.
THE MODEL — who owns what here
| path | owner:group | mode | writable by the www-data group? |
|---|---|---|---|
/var/www | spiffy-root:www-data | 2775 | ✅ |
/var/www/starter | spiffy-root:www-data | 2775 | ✅ |
/var/www/starter/brains | spiffy-root:www-data | 2775 | ✅ |
a created unit (/var/www/{unit}) | www-data:www-data | 02775 (since 2026-08-10) | ✅ |
spiffy-rootis already in thewww-datagroup (gid 33). Nousermodneeded — that was done long ago.sudorequires a password, so an assistant session cannot run it. Anything needing root has to be run by a human.2775is the target mode for directories. The leading2is setgid — it makes everything created inside inherit thewww-datagroup. Without it, new files land in the creator's own group and the web server loses access.- Target modes: directories
2775, files664. Nothing in this tree is legitimately executable — verified: the only+xfiles found were artefacts of bad source modes, not scripts.
Rule of thumb: if you are working inside /var/www/starter/**, you do not need sudo. If you reach for it there, something is already wrong.
THE FOUR THINGS THAT ACTUALLY BREAK IT
1. sudo cp — the one that caused this document
cp derives the new file's mode from the source, and sudo makes it root-owned. A source at 0700 copied with sudo produces root:www-data drwx--S--- at the destination — which the web server cannot even traverse, so the site 403s/500s on files that look present.
Never
sudo cpinto a web tree. Usersyncwith explicit--chown/--chmod.
2. mkdir's mode is UMASK-CLIPPED
Measured 2026-08-10:
| umask | mkdir(0775) produces |
|---|---|
022 — the default; nothing configures otherwise for Apache/PHP-FPM | drwxr-sr-x → 0755, group-write silently lost |
002 | drwxrwsr-x → 0775 ✓ |
So asking for 0775 in code is not enough. Assert it afterwards — this is the pattern the bridge already used at accid-bridge.php:2586:
mkdir($dir, 0775, true);
@chmod($dir, 02775); // mkdir mode is umask-clipped; assert it
3. Default ACLs do NOT rescue an explicit mode
A tempting fix is setfacl -d -m g:www-data:rwx. It does not work for directories created with an explicit mode. Measured 2026-08-10 — default ACL in place, then mkdir(0755):
group:www-data:rwx #effective:r-x
mask::r-x
The requested mode sets the ACL mask, which masks the entry down to r-x. The directory is not group-writable despite the ACL.
- ACLs do help ordinary files (creation requests
0666 & ~umask, so the mask stays permissive). - ACLs cannot widen a mode the creating process narrowed.
Nothing downstream can widen a mode the creator narrowed. Fix it at the
mkdir.
4. rsync --exclude also protects from --delete
An excluded file already present in the destination is not deleted. This bit twice in one session: --exclude accid_project.json on a --delete sync left a stale identity file in place, silently.
rsync -a --delete --exclude accid_project.json SRC/ DEST/
rm -f DEST/accid_project.json # <- the required second step
DELETING
Deleting needs write on the containing directory, not on the file. That is why root-owned 0700 dirs block deletion of files you own.
# a whole unit/subdomain — the DIRECTORY IS THE REGISTRATION.
# Subdomains are wildcard-routed (ServerAlias *.shawns-machine.com,
# VirtualDocumentRoot /var/www/%1) so there is no vhost, no apache reload,
# no registry entry. Remove the folder and the name is free again.
sudo rm -rf /var/www/UNIT
# a project inside an install — no sudo, /var/www/starter/** is 2775
rm -rf /var/www/starter/brains/PROJECT-accid
# QUARANTINE instead of delete — the pattern that has saved this project twice
mv /var/www/starter/brains/PROJECT-accid \
/var/www/starter/_JUNK/PROJECT-purge-$(date +%F)/
Always ls the target first. rm -rf with a typo is the only operation here with no undo.
COPYING
Locally
# contents into — trailing slash on BOTH sides.
# No nesting, and dotfiles come along (which `SRC/*` silently skips).
rsync -a --chown=www-data:www-data --chmod=D2775,F664 SRC/ DEST/
# make DEST *match* SRC (removes anything extra)
rsync -a --delete --chown=www-data:www-data --chmod=D2775,F664 SRC/ DEST/
Why not cp -r SRC DEST: if DEST exists, cp copies into it — you get DEST/SRC/. And SRC/* skips dotfiles, so things like .builder are quietly left behind.
From a desktop, up
# preferred — sets modes, resumes, skips unchanged
rsync -avz --chmod=D2775,F664 \
~/Documents/thing.md \
spiffy-root@shawns-machine.com:/var/www/starter/_JUNK/
# scp uses your local umask, so fix the mode afterwards
scp thing.md spiffy-root@shawns-machine.com:/var/www/starter/_JUNK/
ssh spiffy-root@shawns-machine.com 'chmod 664 /var/www/starter/_JUNK/thing.md'
/var/www/starter/** is setgid, so anything landed there gets group www-data automatically — only the mode needs attention.
Seeding / cloning a project
Identity must never travel. copy_welcome_files() copies everything except an explicit exclude list, and the main call site passes none — so whatever sits in the seed is inherited by every project created from it.
rsync -a --delete \
--exclude accid_project.json \
--exclude accid_secrets.json \
--exclude 'dropper/.accid-backups' \
--exclude 'dropper/.accid-snapshots' \
--chown=www-data:www-data --chmod=D2775,F664 \
SRC/ DEST/
rm -f DEST/accid_project.json DEST/accid_secrets.json # --exclude won't delete these
accid_project.json carries dev_project and an auth block; accid_secrets.json carries the auth hash. A seed containing either mints every new project with a borrowed identity.
FIXING A TREE THAT IS ALREADY BROKEN
Symptom: files exist but the site 403s/500s, or you cannot delete something you appear to own. Check for root-owned or 0700 entries.
# what is wrong
find /var/www/UNIT \( ! -user www-data -o -type d ! -perm -g+w \) -printf '%M %u:%g %p\n' | head
# fix it
sudo chown -R www-data:www-data /var/www/UNIT
sudo find /var/www/UNIT -type d -exec chmod 2775 {} +
sudo find /var/www/UNIT -type f -exec chmod 664 {} +
Two caveats. The blanket chmod 664 flattens anything genuinely executable — check first (find … -perm -u+x) rather than assuming. And .accid-backups / .accid-snapshots are created 0700 on purpose (Apache-private); pulling them into 2775 is a deliberate choice, not a cleanup.
WHAT THE ENGINE NOW DOES (changed 2026-08-10)
accid-bridge.php creates units group-writable, and asserts the mode because mkdir alone cannot:
mkdir(…"/{$dev_project}", 0775, true);
@chmod(…"/{$dev_project}", 02775);
mkdir(…"/{$dev_project}/brains", 0775, true);
@chmod(…"/{$dev_project}/brains", 02775);
Verified end-to-end under Apache's real umask: drwxrwsr-x www-data, writable by the group, no sudo needed to maintain a unit.
This is not a grant to the web server — www-data already owns these. It lets humans and tooling in the www-data group maintain a unit the same way /var/www/starter (2775) has always worked. Deliberately not applied to snap_dir/ms_dir (0700, Apache-private by design).
QUICK REFERENCE
| I want to… | command |
|---|---|
| copy a folder's contents somewhere | rsync -a --chown=www-data:www-data --chmod=D2775,F664 SRC/ DEST/ |
| make DEST match SRC exactly | add --delete (then rm anything you --excluded) |
| upload from the desktop | rsync -avz --chmod=D2775,F664 file host:/path/ |
| delete a subdomain | sudo rm -rf /var/www/UNIT — nothing else registers it |
| delete a project | rm -rf /var/www/starter/brains/PROJECT-accid (no sudo) |
| get rid of something safely | mv it into _JUNK/…-purge-$(date +%F)/ |
| repair a broken tree | chown -R www-data:www-data + 2775 dirs + 664 files |
| check what is wrong | find PATH \( ! -user www-data -o -type d ! -perm -g+w \) -printf '%M %u:%g %p\n' |
The one-line version: directories 2775, files 664, group www-data, never sudo cp, and assert the mode after mkdir because umask will clip it.