Softpanorama May the source be with you, but remember the KISS principle ;-)	Home	Switchboard	Unix Administration	Red Hat	TCP/IP Networks	Neoliberalism	Toxic Managers
	(slightly skeptical) Educational society promoting "Back to basics" movement against IT overcomplexity and bastardization of classic Unix

SQL Links

News

Recommended books

NEWS CONTENTS

20110815 : SQL Tutorial by James Hoffman. ( SQL Tutorial, Aug 15, 2011 )
20110814 : SQLite ( SQLite, Aug 14, 2011 )
20080811 : Reverse Snowflake Joins 0.05 by alxtoth ( Reverse Snowflake Joins 0.05 , Aug 11, 2008 )

Old News ;-)

[Aug 15, 2011] SQL Tutorial by James Hoffman.

New, updated links

A PDF/Adobe Acrobat version of the tutorial is available directly from Matthew Kelly at Highcroft Communications (www.highcroft.com).

Copyright 1996-2001, James Hoffman. This document can be used for free by any Internet user, but cannot be included in another document, another web site or server, published in any other form, or mass produced in any way.

Last updated: 5-14-2001; added relational algebra material.

[Aug 14, 2011] SQLite

The source code for SQLite is in the public domain.

About: SQLite is a C library that implements an SQL database engine. A large subset of SQL92 is supported. A complete database is stored in a single disk file. The API is designed for convenience and ease of use. Applications that link against SQLite can enjoy the power and flexibility of an SQL database without the administrative hassles of supporting a separate database server. Because it omits the client-server interaction overhead and writes directly to disk, SQLite is also faster than the big database servers for most operations. In addition to the C library, the SQLite distribution includes a command-line tool for interacting with SQLite databases and SQLite bindings for Tcl/Tk.

Current Status

Version 3.7.7.1 of SQLite is recommended for all new development. Upgrading from version 3.7.6.3 or 3.7.7 is optional. Upgrading from all other SQLite versions is recommended.

[Aug 11, 2008] Reverse Snowflake Joins 0.05 by alxtoth

About: Reverse Snowflake Joins is a tool that parses SQL Select statements and generates a diagram. In addition to joins, the diagram shows parts of the underlying SQL directly in the diagram. For example x=30, GROUP BY (year), SUM(profit), HAVING MIN(age) > 18.

Changes: The ability to use "select *" and "table.*" were added. The "like" operator was added. Sometimes text was clipped in Graphviz. This was fixed by specifying textsize.

There are three typical implementation models of databases: hierarchical, network, and relational. Each is based on the notion of data stored as a set of records (imagine a set of file cards, for example). Hierarchical (e.g., IMS) and network (e.g., IDMS) models are based on traversing data links to process a database; they are typically used for large mainframe systems and are not considered further here.

We focus on relational database management systems (RDBMSs). They have become popular, perhaps largely due to their simple data model:

Data is presented as a collection of relations
Each relation is depicted as a table
Columns are attributes
Rows ("tuples") represent entities
Every table has a set of attributes that taken together as a "key" (technically, a "superkey") uniquely identifies each entity

For example, a company might have an Employee table with a row for each employee. What attributes might be interesting? This, of course, depends on the application and use the data will be put to, and is determined at database design time. In our example, we might have a payroll application and need salary and mailing address information.

Just as a side note, the notion of view can be useful. Imagine that a company maintains a database of its employees -- there might be a lot of attributes like age, salary, emergency contacts, appraisal, etc. There may be needs to look at the database for different applications serving different users. The company may need to make available demographic data, for example, to a governmental agency. Only some of the attributes need be supplied - and others ought not to so as to protect privacy. Different views can be provided into the same data; in a RDBMS, a view can be seen as yet another table.

ER Data Modeling

Just a few words about design. How do you go about designing a database? It is useful to build a high level conceptual data model where we depict the entities that we are dealing with, their various attributes, and their relationships. An entity is some object with a real or conceptual existence in the world -- "tofu", "Advanced Java Class", "Folger Museum", "Elaine", "company", for example. An attribute is a property of an entity -- "address", "size", "mother", "age", for example. As mentioned above, a relational column is an attribute. A relationship defines roles in which entities work together -- "Bill WORKS-FOR Motorola", "jbs TEACHES advanced-java". RDBMSs represent relationships as tables. A side note for those already familiar with normalizing databases - ER design has been shown (Eugene Wong) to give relations in third normal form. Also, ER diagrams can be mapped not just to RDBMS, but also to the network and hierarchical models.

It is relatively straighforward to represent a database design in graphical ER Diagrams, where rectangles represent entity types, diamonds relationship types, and ovals attributes. Underlined attribute names represent keys. Here is an example ER diagram:

The Relational Algebra vis-a-vis The Relational Calculus

The Relational Algebra

E.F. Codd's work that inspired RDBMSs was based on mathematical notions, so it is no surprise that the theory of database operations are based on set theory. If you are math-averse, don't be "scared" by this section; you can safely skim or skip it, but see if the Select and Project operators make sense, and review the Join diagram.

The Relational Algebra provides a collection of operations to manipulate relations. It supports the notion of a query, or request to retrieve information from a database. There are set operations:

Union: Given two "union compatible" (having the same tuple types; "UC") relations, it returns a new relation consisting of the set unions. ("Howard is the new president of the merged companies A and B and wants to see the total set of his employees: A.EmployeeTable OR B.EmployeeTable")
Intersection: Create a new relation by intersecting two UC relations. ("Amy wants a table of all organizations that are both vegetarian and raw foods in their orientation: A.VegetarianOrganizations AND B.RawFoodOrganizations")
Difference: Return the set difference of two UC relations. ("Laurie wants to look at a table of all restaurants in Chapel Hill that serve vegetarian food but not veal")
Cartesian Product: Not widely used but typically do a Join operation (see below); takes two relations that are not necessarily UC and creates tuples with combined attributes -- R(AttrR1, AttrR2, ... , AttrRi) x S (AttrS1, AttrS2, ... , AttrSj) results in Q with Ri+Sj attributes, Q (AttrS1, AttrS2, ... , AttrSj, AttrR1, ... , AttrRi).

There are also more widely recognized pure database operations. To be sure symbols show up regardless of browser, I will use O for the sigma operator, P for the pi character, and 8 for a "bowtie" (look at the 8 sideways) join operator character.

Select

Ocond (R) applies condition cond to relation R to return a subset of tuples of R. Essentially, you are selecting rows from a table by applying a test based on the relation's attributes. Thus, we can select from our Employee Table all employees where City="Chapel Hill" and (Salary > 100000 OR Salary < 50000). Note that since the entire row is selected, the resulting table still has unique keys.

Project

P attrlist (R) selects columns with attributes in attrlist from relation R. We might have a huge employee table with many attributes we don't want to see, so we can look at a more directed projection of, perhaps, just SSN and salary. (If one of the attributes is not a key, potential duplicates are discarded.)

Join

This is probably the most complex operation, and it consists of a cartesian product followed by a selection on some formula (F in the diagram below). R1 8cond R2 computes a cartesian product of relations R1 and R2 to give an intermediate table (with attributes from both of the input tables), then it applies condition cond to select a subset of rows from the intermediate table. In the illustration below, we work with an abbreviated version of the Employee Table and a new Department Table to find all records about a particular department. From the intermediate table consisting of the 5 x 5 = 25 entries:

      jbs 010-00-1111 A32 A09 Multimedia Projects
      jbs 010-00-1111 A32 A11 Software Reuse
      jbs 010-00-1111 A32 A21 New Department
      jbs 010-00-1111 A32 A32 Java Applications
      jbs 010-00-1111 A32 B01 Accounting
      wms 033-53-3902 A32 A09 Multimedia Projects
      ...
      jbs 505-47-8901 A09 B01 Accounting

we select those records where Dept is A32.

Phew! Just a few more points. We have described the most general join, called a theta join, where the condition can be complex. Typically, the condition is simply testing if a set of attributes equal a set of values (att1 = val1 & att2 = val2 & ...); then we have an equijoin. (The next sentence is false. The previous sentence is true. Just seeing if you're with me!) Finally, a semijoin, also common in practice, is a subset of tuples of the first relation that participate in the join with the second relation; it is represented with a "bowtie" operator where the right-most vertical line is missing. In our example above, were it a semijoin, the result would just be the subset of Employee Table2 consisting of the first two records.

Still with me?! We skipped some additional operators like natural joins (really just a notational shortcut for equijoins), set division (very rarely used and rather awkward!), outer joins, outer unions, and aggregate functions (mathematical functions applied to values in a database - e.g., average age calculated from an age attribute, or count of records).

The basic message I hope that you got is that the relational algebra allows one in a set theoretic fashion to retrieve information from a database. As end users we would probably prefer to be less mathematical, and that's where the Relational Calculus comes in!

The Relational Calculus

The Relational Calculus is a formal query language. Instead of having to write a sequence of relational algebra operations, we simply write a single declarative expression, describing the results that we want. This is somewhat akin to writing a program in C or java instead of assembler, or (in the spirit of real world examples!) telling the babysitter to call with any problems instead of detailing how to pick up the phone, dial numbers, etc.

The expressive power is identical to using relational algebra. Many commercial databases use a language like ... like ... (this is the keyword you were waiting for - sorry you had to wade so far!) SQL (finally!) -- Structured Query Language -- or even a language like QBE (Query by Example) or QUEL (similar to SQL and used for the INGRES RDBMS). A specific relational query language is said to be relationally complete if it can be used to express any query that the relational calculus supports.

There are two common ways of creating a relational calculus (both are based on First Order Predicate Calculus, or basic logical operators). In a Tuple Relational Calculus, variables range over tuples - i.e., variables can take on values of individual table rows. This is just what we want to do a routine query, such as selecting all food items (tuples) from a grocery store (table) where all the ingredients (specific attribute) are organic (value), say. In a Domain Relational Calculus, variables range over domain values of the attributes. This tends to be more complex, and variables are required for each distinct attribute.

But enough theory! In the remainder of the lesson, we'll take a quick look at SQL and then conclude by looking at some Microsoft Access (which uses SQL) screens. Peanut butter and jelly break anybody?

SQL: Data Definition and Data Manipulation Language

SQL is both a Data Definition Language (DDL) and a Data Manipulation Language (DML). As a DDL, it allows a database administrator or database designer to define tables, create views, etc. As a DML, it allows an end user to retrieve information from tables. It came from an IBM Research project entitled "SEQUEL" where the intent was to create a structured English-like query language to interface to the early System R database system. Along with QUEL, SQL was the first high level declarative database language.

In this section, we will just give a few examples of SQL syntax to help suggest some familiarity with the style. For further reference, any number of books can be consulted. Also, SQL is widely used, and a quick search on the web came up with an excellent syntax reference, as well as a pretty good one and one that presents several examples to teach syntax.

Creating and Updating a Database

In this example that follows, we create a table and insert two records. Note that attributes are positional and are specified in the same order in Create Table, unless a specific ordered attribute list is specified in the Insert Into statement (non-specified values are null).

Create Table Song (Title varchar(20) not null, Artist varchar(16) not null, Album varchar(20), Time char(5) ); Insert Into Song Values ("Roundabout", "Yes", "Fragile", "9:35"); Insert Into Song (Time, Artist, Title) Values ("19:35", "Yes!", "I'll be the Roundabout"); Update Employee Set Salary = Salary * 1.2 Where Evaluation > .85;

As you can see, SQL statements look a bit like English. The Delete statement (with a Where clause to specify conditions) removes selected tuples from a table.

Querying from a Database

The Select (no relation to the relational algebra operation) statement is probably the most widely used SQL statement, and it is used to retrieve data from a database. It has many options, and we will again just give a few examples to give a flavor.

The most basic Select statement on, say, a table called Bike, is

Select * From Bike;

This just returns all tuples in the Bike table. We can be more selective and ask for, say, just the attributes Color, Serial Number, and Number of Gears:

Select Color, Serial Number, Number of Gears From Bike;

This essentially applies the select and project relational operators to the table.

We can also apply conditions to be more selective. Maybe we want to look at our inventory of blue bikes with at least 10 gears and see which ones (identified by their serial numbers) have which number of gears, as well as their warehouse location:

Select Serial Number, Number of Gears, Location From Bike Where Color = "Blue" and Number of Gears >= 10;

We can even retrieve from multiple tables. For each blue bike, let's look at its serial number, location, manufacturer's name, and manufacturing date. We assume we have a table Manufacturer which has Serial Number as key and Date and Name as some attributes. To illustrate a point, let's assume that both tables have Name as an attribute; the value in the inventory on-hand Bike table is a vendor-supplied name, while the value of Name in the Manufacturer table is the name of the manufacturer.

Select Serial Number, Location, Manufacturer.Name, Date From Bike, Manufacturer Where Color = "Blue" And Bike.Serial Number = Manufacturer.Serial Number;

(Note that we disambiguated Name by prefixing it with the table name followed by a period.) This example is like a relational algebra select-project-join with equijoin condition on Color.

Let's look at the SQL for the join example we illustrated above. It is fairly straightforward:

Select * From Employee Table2, Department Table Where Dept = "A32";

If we want to look at the distinct salaries we are paying to people in department A32, we can use the Distinct keyword:

Select Distinct Salary From Employee Where Dept = "A32";

These are just a few examples, but I hope that they show the power and relative ease of SQL. It's hard to believe all the theory that we very lightly touched on above lies beneath such straightforward declarative syntax.

Introduction to Access

Microsoft Access provides a graphical user interface that makes it very easy to define and manipulate databases. Let's take a quick peek at a real Access database that I maintain for membership records of an organization that I am involved with.

Access allows you to define and then store a set of queries and give these queries names that are meaningful to you. Note the Tables and Queries tabs in particular (Reports is useful for generating hardcopy output, such as mailing labels).

From this screen, if we select the Design button, we can inspect and modify the query. Access makes it very easy to select records from a database; the user doesn't have to write SQL at all.
If we View the SQL instead of the Query Design, we get something less friendly looking:

SELECT DISTINCTROW [TVS Membership].LAST_NAME, [TVS Membership].FIRST_NAME, [TVS Membership].MEMBER_TYP, [TVS Membership].ADDRESS1, [TVS Membership].ADDRESS2, [TVS Membership].CITY, [TVS Membership].STATE, [TVS Membership].ZIP, [TVS Membership].EXPIRATION FROM [TVS Membership] WHERE ( ( ([TVS Membership].MEMBER_TYP)<>"C" And ([TVS Membership].MEMBER_TYP)<>"1") AND ( ([TVS Membership].EXPIRATION)>Date()-60 And ([TVS Membership].EXPIRATION)< Date() ) ) ORDER BY [TVS Membership].ZIP;

Finally, here we see how we can enter new records in the database. We simply double click the name of our table and go to the last entry, a pseudo- placeholder entry for a new record marked with an asterisk in the left column. We just start typing in the field values, tabbing field-to-field. Here you can see a new record being created for Victor the Vegetarian.

This should give you a general idea of what Access looks like. For more details, I have put together a detailed Access tutorial. It steps you through creating the Employee Table we have been discussing, as well as retrieving data from that table.

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers : Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda : SE quotes : Language Design and Programming Quotes : Random IT-related quotes : Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce : Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 : Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds : Larry Wall : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS : Programming Languages History : PL/1 : Simula 67 : C : History of GCC development : Scripting Languages : Perl history : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month : How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater�s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

Copyright � 1996-2021 by Softpanorama Society. www.softpanorama.org was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

You can use PayPal to to buy a cup of coffee for authors of this site

Disclaimer:

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified:March 12, 2019

Top Visited <p>Your browser does not support iframes.</p>					Switchboard
					Latest
					Past week
					Past month

SQL Links

NEWS CONTENTS

Old News ;-)

[Aug 15, 2011] SQL Tutorial by James Hoffman.

New, updated links

[Aug 14, 2011] SQLite

The source code for SQLite is in the public domain.

[Aug 11, 2008] Reverse Snowflake Joins 0.05 by alxtoth

Recommended Links

Google matched content

Softpanorama Recommended

Top articles

Sites

Getting started

Basics

Reference

The Relational Database Model

ER Data Modeling

The Relational Algebra vis-a-vis The Relational Calculus

The Relational Algebra

The Relational Calculus

SQL: Data Definition and Data Manipulation Language

Creating and Updating a Database

Querying from a Database

Introduction to Access

Etc