It can be a little bit tricky to figure out how to get your robots.txt file to be included in the deployment of your Angular app to Firebase hosting, but it is actually quite simple in the end.

I originally added my robots.txt file to my src folder, but it still wasn't being deployed. If you are running into this issue the steps listed below will help you out.

Add your robots.txt file to the src folder of your Angular Application

If you have never created a robots.txt file, it simply tells search engines (robots/crawlers) which pages of your website should be crawled and which pages should not be crawled.

Here is an example robots.txt file, this example allows any robot to crawl any page on your site:

User-agent: * 
Disallow:

The robots.txt file should be placed along side other src files such as favicon.ico, index.html and sitemap.xml.

SEO Tip

Alongside the robots.txt file, the sitemap.xml file is a must have in order for search engines like Google to properly crawl and index your content.

Want to add a sitemap.xml file to your Angular App?

Check out my article: How to Add a Static sitemap.xml to Angular App with Firebase Hosting

Once you you have created your robots.txt file and placed it in your /src folder, it's time to update your angular.json file.

Add an asset reference to your robots.txt file in angular.json

The next step is to include the path src/robots.txt inside of the assets array within your angular.json. The reference will most likely be listed right after src/favicon.ico and src/assets.

Here is a snippet of my angular.json file that will include my robots.txt when we deploy our project to Firebase hosting:

{
  "$schema": "./node_modules/@angular/cli/lib/config/schema.json",
  "version": 1,
  "newProjectRoot": "projects",
  "projects": {
    "BrendanSlukeDotCom": {
      "projectType": "application",
      "schematics": {
        "@schematics/angular:component": {
          "style": "sass"
        },
        "@schematics/angular:application": {
          "strict": true
        }
      },
      "root": "",
      "sourceRoot": "src",
      "prefix": "app",
      "architect": {
        "build": {
          "builder": "@angular-devkit/build-angular:browser",
          "options": {
            "outputPath": "public",
            "index": "src/index.html",
            "main": "src/main.ts",
            "polyfills": "src/polyfills.ts",
            "tsConfig": "tsconfig.app.json",
            "aot": true,
            "assets": [
              "src/favicon.ico",
              "src/assets",
              "src/robots.txt"
            ],

The rest of the angular.json file is truncated in order to keep the code snippet readable.

Deploy your Angular App to Firebase Hosting

The next step is to deploy your app and see if the robots.txt file is visible and returns a 200 (OK).

Use this command in the Angular CLI to both build and deploy your project with a single command:

ng build --prod | firebase deploy

Your robots.txt file should now be visible at /robots.txt (my robots.txt file is at https://www.brendansluke.com/robots.txt).

Conclusion

I hope this short guide helped you out. If you are looking for additional resources on robots.txt you should check out these resources:

Robots.txt File - Moz

Introduction to robots.txt - Google Search Central

Robots.txt Test Tool - Ryte

FeedbackI'd love to hear your feedback!
Did this blog post help fix your issue? Still having trouble?
Tweet At Me
Brendan SlukeSoftware Engineer

Hi, I'm Brendan Sluke and I love writing code and blogging about software engineering.

Follow Me On Twitter Read My Blog
Recommended Blog PostsBlog Posts I Think You Will Find Interesting
Go Back To Top of Page