Creating 301s for .htaccess

Introduction

Migrating a website to a new URL structure requires careful planning to ensure all old URLs are redirected properly to the new URLs. This process involves recording the current URL structure, matching old URLs to new URLs, and creating redirects in the .htaccess file. This guide will walk you through the steps to accomplish this task.

Analyzing and Recording the Current URL Structure

1. Crawl the Existing Website:
   • Use a tool like Screaming Frog to crawl the website and extract the URL structure.
   • Save the results in a CSV or Excel file for easy reference.
2. Generate a Sitemap:
   • Ensure you have the latest XML sitemap of the website.
   • Use the sitemap to cross-verify the URLs captured by Screaming Frog.
3. Backup the Data:
   • Preserve a copy of the crawl results and sitemap for future reference.

Preparing for the New URL Structure

1. List the New URLs:
   • Obtain the new URL structure from the development team.
   • Ensure all new URLs are finalized before starting the mapping process.
2. Categorize Content:
   • Separate blog URLs from other URLs, as blog URLs might require different handling.

Matching Old URLs to New URLs

1. Mapping Old to New URLs:
   • For each old URL, find the corresponding new URL.
   • Document the matches in a spreadsheet, ensuring every old URL has a corresponding new URL.
   • Note that new URLs do not necessarily need to have a matching old URL.
2. Handling Blog URLs:
   • Create a separate list of old blog URLs.
   • Note these URLs need special handling, typically via a support ticket for content migration.

Tips for Matching URLs:

• 1 to 1 Matching:
  • Direct match where an old URL has a new URL counterpart.
  • Example: /about -> /about-us/
• Many to 1 Matching:
  • Multiple old URLs redirect to a single new URL.
  • Example: /services/3d-printing, /services/digital-scanner -> /services/advanced-dental-technology.html

Creating the .htaccess File for Redirects

1. Basic .htaccess Setup:
   • Start with enabling the rewrite engine:apacheCopy codeRewriteEngine On
2. Redirect Non-www to www (Same Domain):
   • Ensure all non-www traffic is redirected to the www version:apacheCopy codeRewriteCond %{HTTP_HOST} !^www\. [NC] RewriteRule ^(.*)$ https://www.%{HTTP_HOST}/$1 [R=301,L]
3. Redirect Old Domain to New Domain:
   • Redirect all traffic from the old domain to the new domain:apacheCopy codeRewriteCond %{HTTP_HOST} ^old-domain\.com$ [NC] RewriteRule ^(.*)$ https://new-domain.com/$1 [R=301,L]
4. Custom Error Pages:
   • Set up custom error pages:apacheCopy codeErrorDocument 404 /custom_404.html
5. Redirecting Index Page:
   • Redirect index.html to the root:apacheCopy codeRewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /index\.html\ HTTP/ RewriteRule ^index\.html$ / [R=301,L]
6. Specific Redirects for Old to New URLs:
   • Add the specific redirects:apacheCopy code# Example of 1 to 1 Redirect Redirect 301 /about https://www.newdomain.com/about-us/ # Example of Many to 1 Redirect Redirect 301 /services/3d-printing https://www.newdomain.com/advanced-dental-technology.html Redirect 301 /services/digital-scanner https://www.newdomain.com/advanced-dental-technology.html
7. Prevent Access to Sensitive Files:
   • Add security measures to prevent access to sensitive files:apacheCopy code<FilesMatch "\.(htaccess|htpasswd|ini|phps|fla|psd|log|sh)$"> Order Allow,Deny Deny from all </FilesMatch>

Examples of URL Matching:

1 to 1 Matching:

• Old URL: /about
• New URL: /about-us/apacheCopy codeRedirect 301 /about https://www.newdomain.com/about-us/

Many to 1 Matching:

• Old URLs: /services/3d-printing, /services/digital-scanner
• New URL: /services/advanced-dental-technology.htmlapacheCopy codeRedirect 301 /services/3d-printing https://www.newdomain.com/advanced-dental-technology.html Redirect 301 /services/digital-scanner https://www.newdomain.com/advanced-dental-technology.html

Step 5: Testing and Finalizing

1. Test the Redirects:
   • Use tools like curl or online HTTP status checkers to ensure all redirects are working as expected.
   • Verify that no URLs are missed and all redirects point to the correct new URLs.
2. Incremental Testing:
   • Implement and test changes incrementally to catch errors early.
3. Backup and Version Control:
   • Keep backups of the original .htaccess file.
   • Use version control to track changes.

Tools for Crawling and URL Mapping

1. Screaming Frog SEO Spider:
   • This tool can crawl websites and extract URLs, helping you create a complete list of old URLs.
   • It also provides options to compare crawls, which can be useful for mapping old URLs to new URLs.
2. URL Profiler:
   • This tool can help extract various URL data, such as meta information and social metrics, which can aid in the URL mapping process.
   • It integrates with Screaming Frog and other SEO tools for comprehensive analysis.

Tools for Creating .htaccess Redirects

1. htaccess Redirect Generator:
   • Websites like htaccessredirect.net provide simple interfaces to generate .htaccess redirect rules.
   • You can input your old and new URLs, and the tool will generate the necessary .htaccess code.
2. Online Regex Tools:
   • Tools like RegExr and Regex101 can help you create and test regular expressions for your rewrite rules.
   • These tools are useful if you need to create more complex redirect rules based on patterns.

Tools for Testing Redirects

1. Redirect Checker:
   • Tools like Redirect-Checker.org allow you to test your redirects to ensure they are working correctly.
   • You can enter your old URL and see if it correctly redirects to the new URL.
2. HTTP Status Code Checker:
   • Websites like httpstatus.io let you check the HTTP status codes for multiple URLs, ensuring your redirects return the expected 301 status.

Example Workflow Using These Tools

1. Crawl the Existing Site:
   • Use Screaming Frog to crawl the existing website and export the list of old URLs.
2. Generate New URL List:
   • Obtain the list of new URLs from your development team or sitemap.
3. Map Old URLs to New URLs:
   • Use a spreadsheet to manually match old URLs to new URLs, or use URL Profiler for more automated analysis.
4. Create Redirects:
   • Use the htaccess Redirect Generator to create the necessary .htaccess rules based on your URL mappings.
5. Test Redirects:
   • Use Redirect Checker and HTTP Status Code Checker to verify that your redirects are correctly implemented and return the expected status codes.
6. Implement and Validate:
   • Implement the .htaccess file on your server and validate each redirect using tools and manual checks.

By leveraging these tools, you can streamline the process of matching old URLs to new URLs and generating the necessary .htaccess redirects. This approach helps ensure accuracy and efficiency in your URL migration projects.

Example Spreadsheet