Case sensitivity of URLs

Softpanorama site uses case sensitive URLs; Junk bots written for Windows which present URL in low case have problem

• Introduction
• Universal Resource Locators (URLs)
  • Fragment URLs
  • Relative URLs
  • URLs in HTML

Introduction

Domain names are case insensitive according to RFC 4343. The rest of URL is sent to the server via the GET method. This may be case sensitive or not. Sites that are hosted on Windows tend to be case insensitive as the underlying file system is case insensitive. Sites hosted on Unix type systems tend to be case sensitive as their underlying file systems are typically case sensitive. The host name part of the URL is always case insensitive, it's the rest of the path that varies.

Traditionally Web sites which use apache server have case sensitive URLs. In such sites there may be URLs, or parts of URLs, where case doesn't matter (that can be achieved via softlinks), but identifying these may not be easy. Users should always consider that URLs are case-sensitive. Looks like starting around Apache 2.1/2.2, there is a directive  "CheckCaseOnly on" was introduced could turn off the "spelling" feature and leave only the case feature active.  A side effect is this could possibly help us do #1 by detecting those URL's that are bad, assuming it logs those cases where it got activated.

This W3 guideline is reasonable. It simply states that one shouldn't make an assumption on how the server handles the URL you are submitting. It is up to the server how to handle the request URL. Most of web servers are unix/linux and that means most of web servers are case sensitive.  You can mount you Web site a SAMBA share.  Since SAMBA files are case-insensitive, a request from Apache for a file would succeed regardless of the case.

You can however to implement case insensitivity in Apache by using 'CheckSpelling On' from mod_speling. It is also possible  to convert URL to lower case via mod rewrite:

RewriteEngine on
rewritemap lowercase int:tolower
RewriteCond $1 [A-Z]
RewriteRule ^/(.*)$ /${lowercase:$1} [R=301,L] 

On the other hand, Wikipedia is case sensitive except the first character of the title. The URLs https://en.wikipedia.org/wiki/Case_sensitivity  and https://en.wikipedia.org/wiki/case_sensitivity  leads to the same article, but https://en.wikipedia.org/wiki/CASE_SENSITIVITY returns 404.

Universal Resource Locators (URLs)

URLs typically consist of three pieces:

1. The name of the protocol used to transfer the resource over the Web.
2. The name of the machine hosting the resource.
3. The name of the resource itself, given as a path.

Consider the URL that designates the current HTML specification:

http://www.w3.org/TR/WD-html4/cover.html

This URL may be read as follows: Use the HTTP protocol to transfer the data residing on the machine www.w3.org in the file /TR/WD-html4/cover.html

URLs in general are case-sensitive (with the exception of machine names). There may be URLs, or parts of URLs, where case doesn't matter, but identifying these may not be easy. Users should always consider that URLs are case-sensitive. /u

The character set of URLs that appear in HTML is specified in [RFC1738].

Fragment URLs

The URL specification en vigeur at the writing of this document ([RFC1738]) offers a mechanism to refer to a resource, but not to a location within a resource. The Web community has adopted a convention called "fragment URLs" to refer to anchors within an HTML document. A fragment URL ends with "#" followed by an anchor identifier. For instance, here is a fragment URL pointing to an anchor named section_2:

http://somesite.com/html/top.html#section_2

Relative URLs

A relative URL (defined in [RFC1808]) doesn't contain any protocol or machine information, and its path generally refers to an HTML document on the same machine as the current document. Relative URLs may contain relative path components (".." means the parent location) and may be fragment URLs.

Relative URLs may be resolved to full URLs, for example when the user attempts to follow a link from one document to another. [RFC1808] defines the normative algorithm for resolving relative URLs. The following description is for convenience only.

Briefly, a full URL is derived from a relative URL by attaching a "base" part to the relative URL. The base part is a URL that may come from any or all of the following sources:

• HTTP transfer protocol header information (see [RFC2068]).
• Metadata (the META element) in the HEAD section of an HTML document.
• Explicit base path information (the BASE element) in the HEAD section of an HTML document, or the CODEBASE attribute of the APPLET element.

[RFC1808] specifies the precedence among multiple sources of base information. For the purposes of this explanation, the last piece of base information takes precedence over the others and HTTP headers are considered to occur earlier than the document HEAD.

If no explicit base information accompanies the document, the base URL is that which designates the location of the current document.

Given a base URL and a relative URL (that does not begin with a slash), a full URL is derived as follows:

• If the base URL ends with a slash the full URL is derived by appending the relative URL to the base URL. For example, if the base URL is http://nosite.com/dir1/dir2/ and the relative URL is gee.html, the derived URL is http://nosite.com/dir1/dir2/gee.html.
• If the base URL doesn't end with a slash, the last piece of the base URL is considered a resource, so the full URL is derived by appending the relative URL to the parent of the base URL. For example, if the base URL is http://nosite.com/dir1/dir2 and the relative URL is gee.html, the derived URL is http://nosite.com/dir1/gee.html

URLs in HTML

In HTML, URLs play a role in these situations:

• When referring to metadata describing a document (see the HEAD element).
• When citing a external reference (see the Q, BLOCKQUOTE, INS, and DEL elements).
• When including an object directly in a document (see the OBJECT, IMG, MAP, FRAME, and IFRAME elements).
• When linking to another document or program (see the BASE, A, LINK, AREA, FORM, INPUT, SCRIPT, and APPLET elements).

In each case, authors may use a full URL, a fragment URL, or a relative URL. Please consult the section on anchors for more information about links and URLs.

MAILTO URLs

In addition to HTTP URLs, authors might want to include MAILTO URLs (see [RFC1738]) in their documents. MAILTO URLs cause email to be sent to some email address. For instance, the author might create a link that, when activated, causes the user agent to open a mail program with the destination address in the "To:" field.

MAILTO URLs have the following syntax:

mailto:email-address

User agents may support MAILTO URL extensions that are not yet Internet standards (e.g., appending subject information to a URL with the syntax "?Subject=my%20subject" where any space characters are replaced by "%20").

Recommended Links

Softpanorama Recommended

Top articles

Sites

Top articles

Sites

• Case sensitivity - Wikipedia, the free encyclopedia
• http://forums.contribs.org/index.php?topic=6457.0
• Should URL be case sensitive - Stack Overflow
• google - Should URL paths be case sensitive - Webmasters Stack Exchange

... section 2.7.3 http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-25#page-19

The scheme and host are case-insensitive and normally provided in lowercase; all other components are compared in a case-sensitive manner.

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March, 29, 2020