Typing any random subdomain gave me a blank white page, so I built a catchall page
My server has close to ten subdomains attached to it. Each service gets one, like blog, coin, travel, couple, game, taro and fishing, and Apache looks at the incoming domain and passes the request to the right port. I wrote up that setup in an earlier post.
One day I mistyped my blog address on my phone. I ended up on blgo.jaeyonging.com, and instead of an error like "This site can't be reached" I got a blank white page. The tab title, though, was my blog's title. Just to check, I tried something random like asdfas.jaeyonging.com and got the same thing. Any address I typed opened the shell of my blog with nothing inside.
In this post I go over why that happened, and what I thought through while changing it so that unregistered subdomains now land on a page I prepared in advance.
Why did any address open
The cause was in two places. The first was DNS. My domain is managed at Gabia, and since adding a record every time I brought up a service was a chore, at some point I set up a single * record so every subdomain points to my server IP. So even subdomains that don't exist anywhere end up at my server. I masked the server IP, but all three return the same address.
$ dig +short jaeyonging.com xxx.xxx.xxx.xxx $ dig +short asdfas.jaeyonging.com xxx.xxx.xxx.xxx $ dig +short nonexist123.jaeyonging.com xxx.xxx.xxx.xxx
The second was Apache. Name based virtual hosts look at the Host header of the request, or the SNI for HTTPS, and look for a vhost whose ServerName or ServerAlias matches. But when nothing matches, Apache doesn't return an error. It sends the request to the first vhost it read for that port. That is the default vhost.
Which one gets read first is decided by the file name order inside sites-enabled. In my case the files starting with blog were at the front. I never once chose the blog as the default, but just because b comes early in the alphabet, the blog was taking every request that didn't match anything else. Running apache2ctl -S shows the default server for each port, so if you run several subdomains, I recommend checking it once.
So why was the page blank
If the blog had just shown up as usual it would have been less strange, but the page was empty. When I opened the dev tools, the HTML had arrived but the JS file was a 500.
GET https://asdfas.jaeyonging.com/assets/index-e215cf4d.js 500 (Internal Server Error)
It was the CORS setup on the blog server. The scripts Vite builds are loaded with type="module", and module scripts are requested with an Origin header even when they are same origin. The Express cors middleware throws an error when an Origin not on the allowlist comes in, and that turns straight into a 500. A mistyped host would never be on the allowlist, so not a single JS file got through and only the shell was left.
$ curl -s -H 'Origin: https://asdfas.jaeyonging.com' \
http://localhost:5000/assets/AdminBlog-3568ff89.js | head -c 120
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Error: Not allowed by CORScurl doesn't send an Origin, so if you just run it plainly you get a 200. It only breaks in the browser, which is why I spent a while finding the cause at first.
I laid out a few options and weighed them
The easiest one was adding *.jaeyonging.com to the CORS allowlist. But then instead of a blank page, the blog would open on an endless number of addresses. I happened to be in the middle of an AdSense review, so I wanted as few addresses showing the same content as possible. Also, the blog being the default was an accident in the first place. If I later added a service whose name starts with a, from that day on every mistyped address would go to that service. A setup where adding one config file changes something unrelated is bound to confuse me someday.
Removing the wildcard DNS record was another option. Typos would end at the DNS step, which is the cleanest. In exchange, every time I brought up a service I would have to go into Gabia, add a record and wait for it to propagate. I switched to the wildcard precisely because that was a chore, so I didn't want to go back. And for someone who made a typo, it ends with nothing but a browser error page.
I also thought about a 301 redirect to the blog or the root domain. But then infinitely many addresses would all act like aliases of a real address, and there's no guarantee that someone who mistyped a subdomain was heading for the blog. They might have fumbled while typing coin.
So I settled on a landing page that makes it clear the address doesn't exist and shows the list of services that actually do. The person knows best where they were trying to go, so I figured it was right to let them pick.
Making a default vhost on purpose
The Apache side is simpler than it sounds. If the default vhost is decided by file name order, all you need is a vhost whose name sorts first.
# /etc/apache2/sites-available/000-catchall.conf
# Default vhost that takes every request no other vhost matches
# (mistyped subdomains, direct IP access, etc).
# It has to be loaded first in sites-enabled to become the default vhost,
# so don't change the 000- prefix in the file name.
<VirtualHost *:80>
ServerName catchall.jaeyonging.com
ServerAdmin admin@jaeyonging.com
DocumentRoot /var/www/catchall
<Directory /var/www/catchall>
Require all granted
Options -Indexes
AllowOverride None
</Directory>
ProxyRequests Off
# "/" is the landing page, any other path gets the same page with a 404 status
DirectoryIndex index.html
ErrorDocument 404 /index.html
# keep the endless mistyped subdomains out of search indexes
Header always set X-Robots-Tag "noindex, nofollow"
ErrorLog ${APACHE_LOG_DIR}/catchall_error.log
CustomLog ${APACHE_LOG_DIR}/catchall_access.log combined
</VirtualHost>You could say the 000- at the front of the file name is the whole setup. Digits sort before letters, so this file is always read first. I might look at it later and rename it to something prettier, so I nailed it down with a comment.
ServerName catchall.jaeyonging.com isn't an address anyone actually uses. If ServerName is left empty, the vhost inherits the ServerName from the global server config, and on my server that's jaeyonging.com, so it would look like it overlaps with the root domain vhost. To avoid confusion I gave it a separate name that isn't used anywhere.
All the other vhosts hand requests to a Node server with ProxyPass, but this one has Apache serve an HTML file straight from disk. I did that on purpose. If the landing page depended on a Node process, it would go down whenever the server was having trouble. As it is, the page still shows up even if every PM2 process is down.
ProxyRequests Off keeps this vhost from being used as a forward proxy. It's Off by default so it doesn't need to be written, but I spelled it out. In fact, requests like GET http://azenv.net/ do show up in the log. They are probes checking whether my server can be used as a relay to someone else's site.
Where I got confused once: file order
I ran ls to check the order, and it came out differently than I expected.
$ ls /etc/apache2/sites-enabled | head -4 000-catchall.conf 000-catchall-ssl.conf blog.jaeyonging.com.conf blog-ssl.conf $ LC_ALL=C ls /etc/apache2/sites-enabled | head -4 000-catchall-ssl.conf 000-catchall.conf blog-ssl.conf blog.jaeyonging.com.conf
ls sorts by locale rules, which mostly ignore punctuation like hyphens and dots when comparing. Apache, on the other hand, sorts by raw byte values. A hyphen comes before a dot, so in reality blog-ssl.conf is read before blog.jaeyonging.com.conf. To see the order Apache sees, you need to add LC_ALL=C.
This time the port 80 files and the port 443 files didn't overlap, so the result was the same, but if you build a setup that relies on name order you should know about this difference. Trusting plain ls to line up the order could easily leave the wrong vhost as the default.
HTTPS with the wildcard certificate
At first I thought handling port 80 alone might be enough. But the root domain config has HSTS set with includeSubDomains. Any browser that has visited jaeyonging.com even once will open every subdomain over https only from then on. For those browsers an http landing page is simply never seen. So I made a catchall for 443 as well.
# /etc/apache2/sites-available/000-catchall-ssl.conf (excerpt)
<VirtualHost *:443>
ServerName catchall.jaeyonging.com
SSLEngine on
SSLCertificateFile .../_wildcard_.jaeyonging.com_20260201F68CA.all.crt.pem
SSLCertificateKeyFile .../_wildcard_.jaeyonging.com_20260201F68CA.key.pem
DocumentRoot /var/www/catchall
DirectoryIndex index.html
ErrorDocument 404 /index.html
Header always set X-Robots-Tag "noindex, nofollow"
</VirtualHost>The certificate is the same *.jaeyonging.com wildcard the other services use. Whether it's asdfas or helo, any single level subdomain matches the certificate. But a wildcard covers exactly one level. Add one more dot and the certificate no longer matches, so the request never even reaches the landing page.
$ curl -sS https://a.b.jaeyonging.com/ curl: (60) SSL: no alternative certificate subject name matches target host name 'a.b.jaeyonging.com'
There's no way around this short of getting a separate certificate, so I decided to leave it. Two level typos almost never happen anyway.
While checking this, though, I found something else. The blog config has ServerAlias www.blog.jaeyonging.com in it, and that is also two levels, so for the same reason it can't be reached over https. For months I had an alias that nobody could ever get into. It's a line that came along when I made the config by copying another vhost, and I plan to remove it the next time I touch Apache.
What status code to return
This is the part I spent the longest on. The address doesn't exist, so returning 404 for everything seems right. But the subdomain itself is in DNS and the server responds normally. I saw the root of this subdomain as closer to a "landing page" than a "page that doesn't exist".
So the root (/) returns the landing page with a 200, and requests with any path below it return a 404 while still showing the same landing page as the body. The single line ErrorDocument 404 /index.html does that. When a file that doesn't exist is requested, Apache keeps the 404 status code and only swaps the body for index.html.
Of course, returning 200 at the root makes it something like a soft 404 from a search engine's point of view. With wildcard DNS, addresses can be made up endlessly, so once they start getting indexed there's no cleaning it up. That's why I added X-Robots-Tag: noindex, nofollow as a response header. I also put a robots meta tag in the HTML, but setting it as a header also covers responses that aren't HTML, like the favicon. There's a reason I wrote it as Header always too. Without always, the header isn't added to error responses like a 404.
$ curl -s -o /dev/null -w "%{http_code} %{url_effective}\n" https://asdfas.jaeyonging.com/
200 https://asdfas.jaeyonging.com/
$ curl -s -o /dev/null -w "%{http_code} %{url_effective}\n" https://asdfas.jaeyonging.com/abc
404 https://asdfas.jaeyonging.com/abc
$ curl -sI https://asdfas.jaeyonging.com/ | grep -iE '^(HTTP|x-robots|content-type)'
HTTP/2 200
x-robots-tag: noindex, nofollow
content-type: text/htmlThe page is a single static HTML file
The layout is simple. A 404 notice, the address you came in on, and cards for the services that are actually running. Showing the address looks minor, but it's quite useful. People who made a typo usually don't know what they typed, so putting the address right in front of them shows them at once where it went wrong.
The address isn't inserted by the server but by JS. It's a static file, so there's no server side template. Turning on mod_include would let the server insert it, but I didn't want to enable another module just for this one page.
<div class="host" id="host">jaeyonging.com</div>
<script>
(function () {
var el = document.getElementById('host');
if (el && location.hostname) el.textContent = location.hostname;
})();
</script>There's also a reason it goes in with textContent and not innerHTML. The host name is a value the requester gets to choose. Characters like angle brackets will almost never show up in a host name, but I avoid letting a user chosen value be parsed as HTML out of habit. If JS is turned off, the default jaeyonging.com is shown as is.
I also picked which services to show as cards. I deliberately left out monitor, which is my server status board, and preview, which is a private preview of my portfolio. Putting services that aren't meant for other people on a typo page would just be advertising that they exist.
The cards are laid out with grid-template-columns: repeat(auto-fill, minmax(250px, 1fr)), so the rows wrap to fit the screen width on their own. On a phone they show one per row. Most typos happen on phones, so I paid more attention to the mobile view.
Who came in over two weeks
I gave the catchall its own log files. Counting the two weeks that were kept, there were 6,090 requests from 1,517 IPs, leaving out requests from my home router address.
By response code (catchall_access.log + catchall_ssl_access.log, 2 weeks) 403 2,899 blocked by ModSecurity 404 1,132 200 1,022 400 365 405 48
Almost half are 403s. Those are requests ModSecurity blocked, and the paths look like /.env, /.aws/credentials and /.git/config. They are scanners hunting for config files and keys. With wildcard DNS, scanners make up any subdomain they like and knock on it, and all those requests gather here.
I wondered how many people actually made a typo, but that turned out to be hard to tell from the logs alone. Plenty of requests carry a browser UA, but scanners pretend to be browsers too, so you can't count people by UA.
I built this to guide people who made typos, but once it was running, separating the strange requests from the blog log turned out to be the bigger benefit. Before, all of this was mixed into the blog log, and every time I looked at it I had to filter out the scanner requests first.
What still bothers me
The service list is hard coded in the HTML. When I bring up a new service I have to add the card by hand. In fact I brought up a service called ieum in September, and while writing this post I noticed it's still not in the catchall list.
I also thought about generating the list automatically by reading sites-enabled. But as I mentioned above, some services are hidden on purpose, so in the end I'd still need a separate table of which ones to show. With only ten or so services right now, I figured editing it by hand is better.
And it makes me a little uneasy that the default vhost hangs on a single file name. If someone creates a file whose name sorts before 000-, the default changes silently. I did write a comment, but that someone is most likely going to be me a few months from now.
If you have subdomains open through wildcard DNS, I recommend typing a random address once. If you didn't know what shows up there, that is your server's default vhost right now.