history

Antitrust suits led to vertical integration & the IT revolution

business, history, law, tech in changing society, technology / /

From Barry C. Lynn’s “The Case for Breaking Up Wal-Mart” (Harper’s: 24 July 2006):

As the industrial scholar Alfred D. Chandler has noted, the vertically integrated firm — which dominated the American economy for most of the last century — was to a great degree the product of antitrust enforcement. When Theodore Roosevelt began to limit the ability of large companies to grow horizontally, many responded by buying outside suppliers and integrating their operations into vertical lines of production. Many also set up internal research labs to improve existing products and develop new ones. Antitrust law later played a huge role in launching the information revolution. During the Cold War, the Justice Department routinely used antitrust suits to force high-tech firms to share the technologies they had developed. Targeted firms like IBM, RCA, AT&T, and Xerox spilled many thousands of patents onto the market, where they were available to any American competitor for free.

Antitrust suits led to vertical integration & the IT revolution Read More »

Differences between Macintosh & Unix programmers

business, history, tech in changing society, technology / /

From Eric Steven Raymond’s “Problems in the Environment of Unix” (The Art of Unix Programming: 19 September 2003):

Macintosh programmers are all about the user experience. They’re architects and decorators. They design from the outside in, asking first “What kind of interaction do we want to support?” and then building the application logic behind it to meet the demands of the user-interface design. This leads to programs that are very pretty and infrastructure that is weak and rickety. In one notorious example, as late as Release 9 the MacOS memory manager sometimes required the user to manually deallocate memory by manually chucking out exited but still-resident programs. Unix people are viscerally revolted by this kind of mal-design; they don’t understand how Macintosh people could live with it.

By contrast, Unix people are all about infrastructure. We are plumbers and stonemasons. We design from the inside out, building mighty engines to solve abstractly defined problems (like “How do we get reliable packet-stream delivery from point A to point B over unreliable hardware and links?”). We then wrap thin and often profoundly ugly interfaces around the engines. The commands date(1), find(1), and ed(1) are notorious examples, but there are hundreds of others. Macintosh people are viscerally revolted by this kind of mal-design; they don’t understand how Unix people can live with it. …

In many ways this kind of parochialism has served us well. We are the keepers of the Internet and the World Wide Web. Our software and our traditions dominate serious computing, the applications where 24/7 reliability and minimal downtime is a must. We really are extremely good at building solid infrastructure; not perfect by any means, but there is no other software technical culture that has anywhere close to our track record, and it is one to be proud of. …

To non-technical end users, the software we build tends to be either bewildering and incomprehensible, or clumsy and condescending, or both at the same time. Even when we try to do the user-friendliness thing as earnestly as possible, we’re woefully inconsistent at it. Many of the attitudes and reflexes we’ve inherited from old-school Unix are just wrong for the job. Even when we want to listen to and help Aunt Tillie, we don’t know how — we project our categories and our concerns onto her and give her ‘solutions’ that she finds as daunting as her problems.

Differences between Macintosh & Unix programmers Read More »

The first movie theater

business, history, tech in changing society, technology / /

From Adam Goodheart’s “10 Days That Changed History” (The New York Times: 2 July 2006):

APRIL 16, 1902: The Movies

Motion pictures seemed destined to become a passing fad. Only a few years after Edison’s first crude newsreels were screened — mostly in penny arcades, alongside carnival games and other cheap attractions, the novelty had worn off, and Americans were flocking back to live vaudeville.

Then, in spring 1902, Thomas L. Tally opened his Electric Theater in Los Angeles, a radical new venture devoted to movies and other high-tech devices of the era, like audio recordings.

“Tally was the first person to offer a modern multimedia entertainment experience to the American public,” says the film historian Marc Wanamaker. Before long, his successful movie palace produced imitators nationally, which would become known as “nickelodeons.”

The first movie theater Read More »

The day FDR was almost assassinated

history / /

From Adam Goodheart’s “10 Days That Changed History” (The New York Times: 2 July 2006):

FEB. 15, 1933: The Wobbly Chair

It should have been an easy shot: five rounds at 25 feet. But the gunman, Giuseppe Zangara, an anarchist, lost his balance atop a wobbly chair, and instead of hitting President-elect Franklin D. Roosevelt, he fatally wounded the mayor of Chicago, who was shaking hands with F.D.R.

Had Roosevelt been assassinated, his conservative Texas running mate, John Nance Garner, would most likely have come to power. “The New Deal, the move toward internationalism – these would never have happened,” says Alan Brinkley of Columbia University. “It would have changed the history of the world in the 20th century. I don’t think the Kennedy assassination changed things as much as Roosevelt’s would have.”

The day FDR was almost assassinated Read More »

The date Silicon Valley (& Intel) was born

business, history, tech in changing society, technology / /

From Adam Goodheart’s “10 Days That Changed History” (The New York Times: 2 July 2006):

SEPT. 18, 1957: Revolt of the Nerds

Fed up with their boss, eight lab workers walked off the job on this day in Mountain View, Calif. Their employer, William Shockley, had decided not to continue research into silicon-based semiconductors; frustrated, they decided to undertake the work on their own. The researchers — who would become known as “the traitorous eight” — went on to invent the microprocessor (and to found Intel, among other companies). “Sept. 18 was the birth date of Silicon Valley, of the electronics industry and of the entire digital age,” says Mr. Shockley’s biographer, Joel Shurkin.

The date Silicon Valley (& Intel) was born Read More »

Where we are now

history, politics / /

From Gary Kamiya’s “License to lie” (Salon: 23 June 2006):

We are in a peculiar moment, one in which our politicians seem unable to articulate or even grasp the train wreck unfolding in front of them. Someday in the future, if the Democratic Party manages to transform itself from a cowering shadow to something approaching sentience, perhaps what really happened during the Bush era will be publicly debated.

Perhaps then we can ask how it happened that the government of the United States was hijacked by a bullying, fact-averse religious fanatic and his puppetmaster, an evil courtier out of Shakespeare. How we were plunged into a disastrous war simply because a cabal of ideologues and right-wing zealots, operating in autocratic secrecy, decided they wanted war. And how all of the normal workings of a democratic government — objective analysis, checks and balances, transparency — were simply trashed by an administration waving the bloody shirt of “terror.”

Where we are now Read More »

The origin of broadcast journalism

history, technology / /

From Nicholas Lemann’s “The Murrow Doctrine” (The New Yorker: 23 & 30 January 2006: 38-43):

There is a memorable entry in William Shirer’s Berlin Diary in which he describes – as, in effect, something that happened at work one day – the birth of broadcast journalism. It was Sunday, March 13, 1938, the day after Nazi troops entered Austria. Shirer, in London, got a call from CBS headquarters, in New York, asking him to put together a broadcast in which radio correspondents in the major capitals of Europe, led by Shirer’s boss, Edward R. Murrow, who was on the scene in Vienna, would offer a series of live reports on Hitler’s move and the reaction to it.

The origin of broadcast journalism Read More »

Macaulay in 1841: copyright a tax on readers

art, business, history, law, politics / /

From Thomas Babington Macaulay’s “A Speech Delivered In The House Of Commons On The 5th Of February 1841” (Prime Palaver #4: 1 September 2001):

The principle of copyright is this. It is a tax on readers for the purpose of giving a bounty to writers. The tax is an exceedingly bad one; it is a tax on one of the most innocent and most salutary of human pleasures; and never let us forget, that a tax on innocent pleasures is a premium on vicious pleasures. I admit, however, the necessity of giving a bounty to genius and learning. In order to give such a bounty, I willingly submit even to this severe and burdensome tax. Nay, I am ready to increase the tax, if it can be shown that by so doing I should proportionally increase the bounty. My complaint is, that my honourable and learned friend doubles, triples, quadruples, the tax, and makes scarcely any perceptible addition to the bounty.

Macaulay in 1841: copyright a tax on readers Read More »

Macaulay in 1841 on the problems on the copyright monopoly

art, history, law, on writing, politics, tech in changing society / /

From Thomas Babington Macaulay’s “A Speech Delivered In The House Of Commons On The 5th Of February 1841” (Prime Palaver #4: 1 September 2001):

The question of copyright, Sir, like most questions of civil prudence, is neither black nor white, but grey. The system of copyright has great advantages and great disadvantages; and it is our business to ascertain what these are, and then to make an arrangement under which the advantages may be as far as possible secured, and the disadvantages as far as possible excluded. …

We have, then, only one resource left. We must betake ourselves to copyright, be the inconveniences of copyright what they may. Those inconveniences, in truth, are neither few nor small. Copyright is monopoly, and produces all the effects which the general voice of mankind attributes to monopoly. …

I believe, Sir, that I may with safety take it for granted that the effect of monopoly generally is to make articles scarce, to make them dear, and to make them bad. … Thus, then, stands the case. It is good that authors should be remunerated; and the least exceptionable way of remunerating them is by a monopoly. Yet monopoly is an evil. For the sake of the good we must submit to the evil; but the evil ought not to last a day longer than is necessary for the purpose of securing the good. …

For consider this; the evil effects of the monopoly are proportioned to the length of its duration. But the good effects for the sake of which we bear with the evil effects are by no means proportioned to the length of its duration. A monopoly of sixty years produces twice as much evil as a monopoly of thirty years, and thrice as much evil as a monopoly of twenty years. But it is by no means the fact that a posthumous monopoly of sixty years gives to an author thrice as much pleasure and thrice as strong a motive as a posthumous monopoly of twenty years. On the contrary, the difference is so small as to be hardly perceptible. We all know how faintly we are affected by the prospect of very distant advantages, even when they are advantages which we may reasonably hope that we shall ourselves enjoy. But an advantage that is to be enjoyed more than half a century after we are dead, by somebody, we know not by whom, perhaps by somebody unborn, by somebody utterly unconnected with us, is really no motive at all to action. …

Dr Johnson died fifty-six years ago. If the law were what my honourable and learned friend wishes to make it, somebody would now have the monopoly of Dr Johnson’s works. Who that somebody would be it is impossible to say; but we may venture to guess. I guess, then, that it would have been some bookseller, who was the assign of another bookseller, who was the grandson of a third bookseller, who had bought the copyright from Black Frank, the doctor’s servant and residuary legatee, in 1785 or 1786. Now, would the knowledge that this copyright would exist in 1841 have been a source of gratification to Johnson? Would it have stimulated his exertions? Would it have once drawn him out of his bed before noon? Would it have once cheered him under a fit of the spleen? Would it have induced him to give us one more allegory, one more life of a poet, one more imitation of Juvenal? I firmly believe not. I firmly believe that a hundred years ago, when he was writing our debates for the Gentleman’s Magazine, he would very much rather have had twopence to buy a plate of shin of beef at a cook’s shop underground. Considered as a reward to him, the difference between a twenty years’ and sixty years’ term of posthumous copyright would have been nothing or next to nothing. But is the difference nothing to us? I can buy Rasselas for sixpence; I might have had to give five shillings for it. I can buy the Dictionary, the entire genuine Dictionary, for two guineas, perhaps for less; I might have had to give five or six guineas for it. Do I grudge this to a man like Dr Johnson? Not at all. Show me that the prospect of this boon roused him to any vigorous effort, or sustained his spirits under depressing circumstances, and I am quite willing to pay the price of such an object, heavy as that price is. But what I do complain of is that my circumstances are to be worse, and Johnson’s none the better; that I am to give five pounds for what to him was not worth a farthing.

Macaulay in 1841 on the problems on the copyright monopoly Read More »

The real purposes of the American school

business, education, history, politics / /

From John Taylor Gatto’s “Against School” (Harper’s Magazine: September 2003):

Mass schooling of a compulsory nature really got its teeth into the United States between 1905 and 1915, though it was conceived of much earlier and pushed for throughout most of the nineteenth century. The reason given for this enormous upheaval of family life and cultural traditions was, roughly speaking, threefold:

1) To make good people.
2) To make good citizens.
3) To make each person his or her personal best.

These goals are still trotted out today on a regular basis, and most of us accept them in one form or another as a decent definition of public education’s mission, however short schools actually fall in achieving them. But we are dead wrong. Compounding our error is the fact that the national literature holds numerous and surprisingly consistent statements of compulsory schooling’s true purpose. We have, for example, the great H. L. Mencken, who wrote in The American Mercury for April 1924 that the aim of public education is not

to fill the young of the species with knowledge and awaken their intelligence. . . . Nothing could be further from the truth. The aim.. . is simply to reduce as many individuals as possible to the same safe level, to breed and train a standardized citizenry, to put down dissent and originality. That is its aim in the United States . . . and that is its aim everywhere else. …

[Alexander Inglis, author of the 1918 book, Principles of Secondary Education,], for whom a lecture in education at Harvard is named, makes it perfectly clear that compulsory schooling on this continent was intended to be just what it had been for Prussia in the 1820s: a fifth column into the burgeoning democratic movement that threatened to give the peasants and the proletarians a voice at the bargaining table. Modern, industrialized, compulsory schooling was to make a sort of surgical incision into the prospective unity of these underclasses. Divide children by subject, by age-grading, by constant rankings on tests, and by many other more subtle means, and it was unlikely that the ignorant mass of mankind, separated in childhood, would ever reintegrate into a dangerous whole.

Inglis breaks down the purpose – the actual purpose – of modem schooling into six basic functions, any one of which is enough to curl the hair of those innocent enough to believe the three traditional goals listed earlier:

1) The adjustive or adaptive function. Schools are to establish fixed habits of reaction to authority. This, of course, precludes critical judgment completely. It also pretty much destroys the idea that useful or interesting material should be taught, because you can’t test for reflexive obedience until you know whether you can make kids learn, and do, foolish and boring things.

2) The integrating function. This might well be called “the conformity function,” because its intention is to make children as alike as possible. People who conform are predictable, and this is of great use to those who wish to harness and manipulate a large labor force.

3) The diagnostic and directive function. School is meant to determine each student’s proper social role. This is done by logging evidence mathematically and anecdotally on cumulative records. As in “your permanent record.” Yes, you do have one.

4) The differentiating function. Once their social role has been “diagnosed,” children are to be sorted by role and trained only so far as their destination in the social machine merits – and not one step further. So much for making kids their personal best.

5) The selective function. This refers not to human choice at all but to Darwin’s theory of natural selection as applied to what he called “the favored races.” In short, the idea is to help things along by consciously attempting to improve the breeding stock. Schools are meant to tag the unfit – with poor grades, remedial placement, and other punishments – clearly enough that their peers will accept them as inferior and effectively bar them from the reproductive sweepstakes. That’s what all those little humiliations from first grade onward were intended to do: wash the dirt down the drain.

6) The propaedeutic function. The societal system implied by these rules will require an elite group of caretakers. To that end, a small fraction of the kids will quietly be taught how to manage this continuing project, how to watch over and control a population deliberately dumbed down and declawed in order that government might proceed unchallenged and corporations might never want for obedient labor. …

Class may frame the proposition, as when Woodrow Wilson, then president of Princeton University, said the following to the New York City School Teachers Association in 1909: “We want one class of persons to have a liberal education, and we want another class of persons, a very much larger class, of necessity, in every society, to forgo the privileges of a liberal education and fit themselves to perform specific difficult manual tasks.” …

Now, you needn’t have studied marketing to know that there are two groups of people who can always be convinced to consume more than they need to: addicts and children. School has done a pretty good job of turning our children into addicts, but it has done a spectacular job of turning our children into children. Again, this is no accident. Theorists from Plato to Rousseau to our own Dr. Inglis knew that if children could be cloistered with other children, stripped of responsibility and independence, encouraged to develop only the trivializing emotions of greed, envy, jealousy, and fear, they would grow older but never truly grow up. …

Now for the good news. Once you understand the logic behind modern schooling, its tricks and traps are fairly easy to avoid. School trains children to be employees and consumers; teach your own to be leaders and adventurers. School trains children to obey reflexively; teach your own to think critically and independently. Well-schooled kids have a low threshold for boredom; help your own to develop an inner life so that they’ll never be bored. Urge them to take on the serious material, the grown-up material, in history, literature, philosophy, music, art, economics, theology – all the stuff schoolteachers know well enough to avoid. Challenge your kids with plenty of solitude so that they can learn to enjoy their own company, to conduct inner dialogues. Well-schooled people are conditioned to dread being alone, and they seek constant companionship through the TV, the computer, the cell phone, and through shallow friendships quickly acquired and quickly abandoned. Your children should have a more meaningful life, and they can.

First, though, we must wake up to what our schools really are: laboratories of experimentation on young minds, drill centers for the habits and attitudes that corporate society demands. Mandatory education serves children only incidentally; its real purpose is to turn them into servants. Don’t let your own have their childhoods extended, not even for a day. If David Farragut could take command of a captured British warship as a preteen, if Thomas Edison could publish a broadsheet at the age of twelve, if Ben Franklin could apprentice himself to a printer at the same age (then put himself through a course of study that would choke a Yale senior today), there’s no telling what your own kids could do. After a long life, and thirty years in the public school trenches, I’ve concluded that genius is as common as dirt. We suppress our genius only because we haven’t yet figured out how to manage a population of educated men and women. The solution, I think, is simple and glorious. Let them manage themselves.

The real purposes of the American school Read More »

The birth of Geology & gradualism as a paradigm shift from catastrophism

history, religion, science, technology / /

From Kim Stanley Robinson’s “Imagining Abrupt Climate Change : Terraforming Earth” (Amazon Shorts: 31 July 2005):

This view, by the way, was in keeping with a larger and older paradigm called gradualism, the result of a dramatic and controversial paradigm shift of its own from the nineteenth century, one that is still a contested part of our culture wars, having to do with the birth of geology as a field, and its discovery of the immense age of the Earth. Before that, Earth’s history tended to be explained in a kind of Biblical paradigm, in which the Earth was understood to be several thousand years old, because of genealogies in the Bible, so that landscape features tended to be explained by events like Noah’s flood. This kind of “catastrophism” paradigm was what led Josiah Whitney to maintain that Yosemite Valley must have been formed by a cataclysmic earthquake, for instance; there simply hadn’t been time for water and ice to have carved something as hard as granite. It was John Muir who made the gradualist argument for glacial action over millions of years; and the eventual acceptance of his explanation was part of the general shift to gradualist explanations for Earth’s landforms, which also meant there was another time for evolution to have taken place. Gradualism also led by extension to thinking that the various climate regimes of the past had also come about fairly gradually.

The birth of Geology & gradualism as a paradigm shift from catastrophism Read More »

Why structureless is not only impossible, but counterproductive

history, politics / /

From Jo Freeman’s “The Tyranny of Structurelessness” (1970):

During the years in which the women’s liberation movement has been taking shape, a great emphasis has been placed on what are called leaderless, structureless groups as the main form of the movement. …

The idea of ‘structurelessness’, however, has moved from a healthy counter to these tendencies to becoming a goddess in its own right. The idea is as little examined as the term is much used, but it has become an intrinsic and unquestioned part of women’s liberation ideology. …

If the movement is to move beyond these elementary stages of development, it will have to disabuse itself of some of its prejudices about organisation and structure. There is nothing inherently bad about either of these. …

Contrary to what we would like to believe, there is no such thing as a ‘structureless’ group. Any group of people of whatever nature coming together for any length of time, for any purpose, will inevitably structure itself in some fashion. The structure may be flexible, it may vary over time, it may evenly or unevenly distribute tasks, power and resources over the members of the group. But it will be formed regardless of the abilities, personalities and intentions of the people involved. The very fact that we are individuals with different talents, predisposition’s and backgrounds makes this inevitable. Only if we refused to relate or interact on any basis whatsoever could we approximate ‘structurelessness’ and that is not the nature of a human group. …

Thus ‘structurelessness’ becomes a way of masking power, and within the women’s movement it is usually most strongly advocated by those who are the most powerful (whether they are conscious of their power or not). The rules of how decisions are made are known only to a few and awareness of power is curtailed by those who know the rules, as long as the structure of the group is informal. Those who do not know the rules and are not chosen for initiation must remain in confusion, or suffer from paranoid delusions that something is happening of which they are not quite aware. …

A structured group always has a formal structure, and may also have an informal one. An unstructured group always has an informal , or covert, structure. It is this informal structure, particularly in unstructured groups, which forms the basis for elites. …

Correctly, an elite refers to a small group of people who have power over a larger group of which they are part, usually without direct responsibility to that larger group, and often without their knowledge or consent. A person becomes an elitist by being part of, or advocating, the rule by such a small group, whether or not that individual is well-known or not known at all. Notoriety is not a definition of an elitist. The most insidious elites are usually run by people not known to the larger public at all. Intelligent elitists are usually smart enough not to allow themselves to become well- known. When they become known, they are watched, and the mask over their power is no longer firmly lodged. …

Only three techniques have ever been developed for establishing mass group opinion: the vote or referendum, the public opinion survey questionnaire and the selection of group spokespeople at an appropriate meeting. The women’s liberation movement has used none of these to communicate with the public. Neither the movement as a whole nor most of the multitudinous groups within it have established a means of explaining their position on various issues. But the public is conditioned to look for spokespeople. …

The more unstructured a movement is, the less control it has over the directions in which it develops and the political actions in which it engages. This does not mean that its ideas do not spread. Given a certain amount of interest by the media and the appropriateness of social conditions, the ideas will still be diffused widely. But diffusion of ideas does not mean they are implemented; it only means they are talked about. Insofar as they can be applied individually they may be acted upon; insofar as they require co-ordinated political power to be implemented, they will not be.

Why structureless is not only impossible, but counterproductive Read More »

America the aggressive

history, politics / /

From Harold Pinter’s “Nobel Lecture: Art, Truth & Politics” (Nobel Prize: 7 December 2005):

Direct invasion of a sovereign state has never in fact been America’s favoured method. In the main, it has preferred what it has described as ‘low intensity conflict’. Low intensity conflict means that thousands of people die but slower than if you dropped a bomb on them in one fell swoop. It means that you infect the heart of the country, that you establish a malignant growth and watch the gangrene bloom. When the populace has been subdued – or beaten to death – the same thing – and your own friends, the military and the great corporations, sit comfortably in power, you go before the camera and say that democracy has prevailed. This was a commonplace in US foreign policy in the years to which I refer. …

The United States supported and in many cases engendered every right wing military dictatorship in the world after the end of the Second World War. I refer to Indonesia, Greece, Uruguay, Brazil, Paraguay, Haiti, Turkey, the Philippines, Guatemala, El Salvador, and, of course, Chile. The horror the United States inflicted upon Chile in 1973 can never be purged and can never be forgiven.

Hundreds of thousands of deaths took place throughout these countries. Did they take place? And are they in all cases attributable to US foreign policy? The answer is yes they did take place and they are attributable to American foreign policy. But you wouldn’t know it.

It never happened. Nothing ever happened. Even while it was happening it wasn’t happening. It didn’t matter. It was of no interest. The crimes of the United States have been systematic, constant, vicious, remorseless, but very few people have actually talked about them. You have to hand it to America. It has exercised a quite clinical manipulation of power worldwide while masquerading as a force for universal good. It’s a brilliant, even witty, highly successful act of hypnosis.

I put to you that the United States is without doubt the greatest show on the road. Brutal, indifferent, scornful and ruthless it may be but it is also very clever. As a salesman it is out on its own and its most saleable commodity is self love. It’s a winner. Listen to all American presidents on television say the words, ‘the American people’, as in the sentence, ‘I say to the American people it is time to pray and to defend the rights of the American people and I ask the American people to trust their president in the action he is about to take on behalf of the American people.’

It’s a scintillating stratagem. Language is actually employed to keep thought at bay. The words ‘the American people’ provide a truly voluptuous cushion of reassurance. You don’t need to think. Just lie back on the cushion. The cushion may be suffocating your intelligence and your critical faculties but it’s very comfortable. This does not apply of course to the 40 million people living below the poverty line and the 2 million men and women imprisoned in the vast gulag of prisons, which extends across the US. …

We have brought torture, cluster bombs, depleted uranium, innumerable acts of random murder, misery, degradation and death to the Iraqi people and call it ‘bringing freedom and democracy to the Middle East’. …

I have said earlier that the United States is now totally frank about putting its cards on the table. That is the case. Its official declared policy is now defined as ‘full spectrum dominance’. That is not my term, it is theirs. ‘Full spectrum dominance’ means control of land, sea, air and space and all attendant resources.

The United States now occupies 702 military installations throughout the world in 132 countries, with the honourable exception of Sweden, of course. We don’t quite know how they got there but they are there all right.

The United States possesses 8,000 active and operational nuclear warheads. Two thousand are on hair trigger alert, ready to be launched with 15 minutes warning. It is developing new systems of nuclear force, known as bunker busters. … We must remind ourselves that the United States is on a permanent military footing and shows no sign of relaxing it.

America the aggressive Read More »

Why software is difficult to create … & will always be difficult

business, history, technology / /

From Frederick P. Brooks, Jr.’s “No Silver Bullet: Essence and Accidents of Software Engineering” (Computer: Vol. 20, No. 4 [April 1987] pp. 10-19):

The familiar software project, at least as seen by the nontechnical manager, has something of this character; it is usually innocent and straightforward, but is capable of becoming a monster of missed schedules, blown budgets, and flawed products. So we hear desperate cries for a silver bullet–something to make software costs drop as rapidly as computer hardware costs do.

But, as we look to the horizon of a decade hence, we see no silver bullet. There is no single development, in either technology or in management technique, that by itself promises even one order-of-magnitude improvement in productivity, in reliability, in simplicity. …

The essence of a software entity is a construct of interlocking concepts: data sets, relationships among data items, algorithms, and invocations of functions. This essence is abstract in that such a conceptual construct is the same under many different representations. It is nonetheless highly precise and richly detailed.

I believe the hard part of building software to be the specification, design, and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation. We still make syntax errors, to be sure; but they are fuzz compared with the conceptual errors in most systems. …

Let us consider the inherent properties of this irreducible essence of modern software systems: complexity, conformity, changeability, and invisibility.

Complexity. Software entities are more complex for their size than perhaps any other human construct because no two parts are alike (at least above the statement level). …

Many of the classic problems of developing software products derive from this essential complexity and its nonlinear increases with size. From the complexity comes the difficulty of communication among team members, which leads to product flaws, cost overruns, schedule delays. From the complexity comes the difficulty of enumerating, much less understanding, all the possible states of the program, and from that comes the unreliability. From complexity of function comes the difficulty of invoking function, which makes programs hard to use. From complexity of structure comes the difficulty of extending programs to new functions without creating side effects. From complexity of structure come the unvisualized states that constitute security trapdoors.

Not only technical problems, but management problems as well come from the complexity. It makes overview hard, thus impeding conceptual integrity. It makes it hard to find and control all the loose ends. It creates the tremendous learning and understanding burden that makes personnel turnover a disaster.

Conformity. … No such faith comforts the software engineer. Much of the complexity that he must master is arbitrary complexity, forced without rhyme or reason by the many human institutions and systems to which his interfaces must conform. …

Changeability. … All successful software gets changed. Two processes are at work. First, as a software product is found to be useful, people try it in new cases at the edge of or beyond the original domain. The pressures for extended function come chiefly from users who like the basic function and invent new uses for it.

Second, successful software survives beyond the normal life of the machine vehicle for which it is first written. If not new computers, then at least new disks, new displays, new printers come along; and the software must be conformed to its new vehicles of opportunity. …

Invisibility. Software is invisible and unvisualizable. …

The reality of software is not inherently embedded in space. Hence, it has no ready geometric representation in the way that land has maps, silicon chips have diagrams, computers have connectivity schematics. As soon as we attempt to diagram software structure, we find it to constitute not one, but several, general directed graphs superimposed one upon another. The several graphs may represent the flow of control, the flow of data, patterns of dependency, time sequence, name-space relationships. These graphs are usually not even planar, much less hierarchical. …

Past Breakthroughs Solved Accidental Difficulties

If we examine the three steps in software technology development that have been most fruitful in the past, we discover that each attacked a different major difficulty in building software, but that those difficulties have been accidental, not essential, difficulties. …

High-level languages. Surely the most powerful stroke for software productivity, reliability, and simplicity has been the progressive use of high-level languages for programming. …

What does a high-level language accomplish? It frees a program from much of its accidental complexity. …

Time-sharing. Time-sharing brought a major improvement in the productivity of programmers and in the quality of their product, although not so large as that brought by high-level languages.

Time-sharing attacks a quite different difficulty. Time-sharing preserves immediacy, and hence enables one to maintain an overview of complexity. …

Unified programming environments. Unix and Interlisp, the first integrated programming environments to come into widespread use, seem to have improved productivity by integral factors. Why?

They attack the accidental difficulties that result from using individual programs together, by providing integrated libraries, unified file formats, and pipes and filters. As a result, conceptual structures that in principle could always call, feed, and use one another can indeed easily do so in practice.

Why software is difficult to create … & will always be difficult Read More »

Paradigm shifts explained

history, science, tech in changing society / /

From Kim Stanley Robinson’s “Imagining Abrupt Climate Change : Terraforming Earth” (Amazon Shorts: 31 July 2005):

… paradigm shifts are exciting moments in science’s ongoing project of self-improvement, making itself more accurately mapped to reality as it is discovered and teased out; this process of continual recalibration and improvement is one of the most admirable parts of science, which among other things is a most powerful and utopian set of mental habits; an attitude toward reality that I have no hesitation in labeling a kind of worship or devotion. And in this ongoing communal act of devotion, paradigm shifts are very good at revealing how science is conducted, in part because each one represents a little (or big) crisis of understanding.

As Thomas Kuhn described the process in his seminal book The Structure of Scientific Revolutions, workers in the various branches of science build over time an interconnected construct of concepts and beliefs that allow them to interpret the data from their experiments, and fit them into a larger picture of the world that makes the best sense of the evidence at hand. What is hoped for is a picture that, if anyone else were to question it, and follow the train of reasoning and all the evidence used to support it, they too would agree with it. This is one of the ways science is interestingly utopian; it attempts to say things that everyone looking at the same evidence would agree to.

So, using this paradigm, always admitted to be a work in progress, scientists then conduct what Kuhn calls “normal science,” elucidating further aspects of reality by using the paradigm to structure their questions and their answers. Sometimes paradigms are useful for centuries; other times, for shorter periods. Then it often happens that scientists in the course of doing “normal science” begin to get evidence from the field that cannot be explained within the paradigm that has been established. At first such “anomalies” are regarded as suspect in themselves, precisely because they don’t fit the paradigm. They’re oddities, and something might be wrong with them as such. Thus they are ignored, or tossed aside, or viewed with suspicion, or in some other way bracketed off. Eventually, if enough of them pile up, and they seem similar in kind, or otherwise solid as observations, attempts might be made to explain them within the old paradigm, by tweaking or re-interpreting the paradigm itself, without actually throwing the paradigm out entirely.

For instance, when it was found that Newtonian laws of gravitation could not account for the speed of Mercury, which was moving a tiny bit faster than it ought to have been, even though Newton’s laws accounted for all the other planets extremely well, at first some astronomers suggested there might be another planet inside the orbit of Mercury, too close to the Sun for us to see. They even gave this potential planet a name, Vulcan; but they couldn’t see it, and calculations revealed that this hypothetical Vulcan still would not explain the discrepancy in Mercury’s motion. The discrepancy remained an anomaly, and was real enough and serious enough to cast the whole Newtonian paradigm into doubt among the small group of people who worried about it and wondered what could be causing it.

It was Einstein who then proposed that Mercury moved differently than predicted because spacetime itself curved around masses, and near the huge mass of the Sun the effect was large enough to be noticeable.

Whoah! This was a rather mind-bogglingly profound explanation for a little orbital discrepancy in Mercury; but Einstein also made a new prediction and suggested an experiment; if his explanation were correct, then light too would bend in the gravity well around the sun, and so the light of a star would appear from behind the sun a little bit before the astronomical tables said that it should. The proposed experiment presented some observational difficulties, but a few years later it was accomplished during a total eclipse of the sun, and the light of a certain star appeared before it ought to have by just the degree Einstein had predicted. And so Einstein’s concepts concerning spacetime began to be accepted and elaborated, eventually forming a big part of the paradigm known as the “standard model,” within which new kinds of “normal science” in physics and astronomy could be done. …

Paradigm shifts explained Read More »

Origins of the interstate highway system

commonplace book, history / /

From Robert Sullivan’s “An Impala’s-Eye View of Highway History” (The New York Times: 14 July 2006):

Another traveler, Dwight D. Eisenhower, spent two months in 1919 driving a military convoy across the country; the shoddy roads left a lasting impression on him. After World War II he studied Hitler’s autobahn and concluded that the American military should have one. In 1956 he signed the Federal-Aid Highway Act, which, the president recounted in his memoir, resulted in enough concrete to build “six sidewalks to the moon.” The new highways were originally meant to loop around cities that could be skirted should they be destroyed by atomic bombs. Instead, the loops started a suburban construction boom that continues to this day. [Robert] Sullivan reports that Phoenix, a city that virtually rose out of the Interstate, currently gobbles up land at the rate of 1.2 acres per hour.

In the 1960’s state toll roads entered into the system, extending the web to all corners of the country. Today almost 47,000 miles of Interstate highways – with attendant motels, fast-food courts and construction projects – have paved over the continent with such efficiency that one can move from sea to shining sea with speed, economy and almost zero interpersonal interaction.

Origins of the interstate highway system Read More »

NSA spying: Project Shamrock & Echelon

history, politics, security, tech in changing society, technology / /

From Kim Zetter’s “The NSA is on the line — all of them” (Salon: 15 May 2006):

As fireworks showered New York Harbor [in 1976], the country was debating a three-decades-long agreement between Western Union and other telecommunications companies to surreptitiously supply the NSA, on a daily basis, with all telegrams sent to and from the United States. The similarity between that earlier program and the most recent one is remarkable, with one exception — the NSA now owns vastly improved technology to sift through and mine massive amounts of data it has collected in what is being described as the world’s single largest database of personal information. And, according to Aid, the mining goes far beyond our phone lines.

The controversy over Project Shamrock in 1976 ultimately led Congress to pass the 1978 Foreign Intelligence Surveillance Act and other privacy and communication laws designed to prevent commercial companies from working in cahoots with the government to conduct wholesale secret surveillance on their customers. But as stories revealed last week, those safeguards had little effect in preventing at least three telecommunications companies from repeating history. …

[Intelligence historian Matthew Aid] compared the agency’s current data mining to Project Shamrock and Echelon, the code name for an NSA computer system that for many years analyzed satellite communication signals outside the U.S., and generated its own controversy when critics claimed that in addition to eavesdropping on enemy communication, the satellites were eavesdropping on allies’ domestic phone and e-mail conversations. …

If you want some historical perspective look at Operation Shamrock, which collapsed in 1975 because [Rep.] Bella Abzug [D-NY] subpoenaed the heads of Western Union and the other telecommunications giants and put them in witness chairs, and they all admitted that they had cooperated with the NSA for the better part of 40 years by supplying cables and telegrams.

The newest system being added to the NSA infrastructure, by the way, is called Project Trailblazer, which was initiated in 2002 and which was supposed to go online about now but is fantastically over budget and way behind schedule. Trailblazer is designed to copy the new forms of telecommunications — fiber optic cable traffic, cellphone communication, BlackBerry and Internet e-mail traffic. …

Echelon, in fact, is nothing more than a VAX microcomputer that was manufactured in the early 1970s by Digital Equipment Corp., and was used at six satellite intercept stations [to filter and sort data collected from the satellites and distribute it to analysts]. The computer has long since been obsolete. Since 9/11, whatever plans in place to modernize Echelon have been put on hold. The NSA does in fact have a global intercept network, but they just call it the intercept collection infrastructure. They don’t have a code name or anything sexy to describe it, and it didn’t do domestic spying.

NSA spying: Project Shamrock & Echelon Read More »

Learn by working on hard problems

business, education, history / /

From Paul Graham’s “Undergraduation” (March 2005):

Thomas Huxley said “Try to learn something about everything and everything about something.” Most universities aim at this ideal.

But what’s everything? To me it means, all that people learn in the course of working honestly on hard problems. …

Working on hard problems is not, by itself, enough. Medieval alchemists were working on a hard problem, but their approach was so bogus that there was little to learn from studying it, except possibly about people’s ability to delude themselves.

Learn by working on hard problems Read More »

Education teaches people how to solve problems, not choose the good ones

business, education, history / /

From Paul Graham’s “Why Smart People Have Bad Ideas” (April 2005):

Why did so few applicants really think about what customers want? I think the problem with many, as with people in their early twenties generally, is that they’ve been trained their whole lives to jump through predefined hoops. They’ve spent 15-20 years solving problems other people have set for them. And how much time deciding what problems would be good to solve? Two or three course projects? They’re good at solving problems, but bad at choosing them.

But that, I’m convinced, is just the effect of training. Or more precisely, the effect of grading. To make grading efficient, everyone has to solve the same problem, and that means it has to be decided in advance. It would be great if schools taught students how to choose problems as well as how to solve them, but I don’t know how you’d run such a class in practice.

Education teaches people how to solve problems, not choose the good ones Read More »

Conservatives are more ruthless than liberals

history, politics / /

From Alan Wolfe’s “Why Conservatives Can’t Govern” (The Washington Monthly: July/August 2006):

Political parties expend the time and grueling energy to control government for different reasons. Liberals, while enjoying the perquisites of office, also want to be in a position to use government to solve problems. But conservatives have different motives for wanting power. One is to prevent liberals from doing so; if government cannot be made to disappear, at least it can be prevented from doing any good. The other is to build a political machine in which business and the Republican Party can exchange mutual favors; business will lavish cash on politicians (called campaign contributions) while politicians will throw the money back at business (called public policy). …

Historically and philosophically, liberals and conservatives have disagreed with each other, not only over the ends political systems should serve, but over the means chosen to serve those ends. Whether through the ideas of James Madison, Immanuel Kant, or John Stuart Mill, liberals have viewed violent conflict as regrettable and the use of political institutions as the best way to contain it. Conservatives, from the days of Machiavelli to such twentieth-century figures as Germany’s Carl Schmitt, have, by contrast, viewed politics as an extension of war, complete with no-holds-barred treatment of the enemy, iron-clad discipline in the ranks, cries of treason against those who do not support the effort with full-throated vigor, and total control over any spoils won. From a conservative point of view, separation of powers is divisive, tolerance a luxury, fairness another word for weakness, and cooperation unnecessary. If conservatives will not use government to tame Hobbes’ state of nature, they will use it to strengthen Hobbes’ state of nature. Victory is the only thing that matters, and any tactic more likely to produce victory is justified.

The K Street Project, then, did not arise spontaneously out of the ether. When Republicans in Congress began to inform lobbyists that in return for influence they would have to fire all the Democrats in their firms, they may have broken with long-standing traditions, but they were simply carrying forward politics-as-warfare the way conservative political philosophers have historically understood it. Liberals do not generally have objections to working with conservatives; indeed, having conservatives sign off on any expansion of government adds to the legitimacy of that expansion. But conservatives tend to see working with liberals as corrupting; in the immortal words of conservative activist Grover Norquist, “bipartisanship is another name for date rape.” K Street is to lobbying what Fox News is to journalistic objectivity. In the world that contemporary conservatives have brought into being, rules are not applicable to all parties to a conflict. Rules are part of the conflict, and whoever wins the conflict gets to change the rules.

Conservatives are more ruthless than liberals Read More »