analysis

An analysis of botnets

From The Honeynet Project & Research Alliance’s “Know your Enemy: Tracking Botnets” (13 March 2005):

A botnet is a network of compromised machines that can be remotely controlled by an attacker. … With the help of honeynets we can observe the people who run botnets … Due to the wealth of data logged, it is possible to reconstruct the actions of attackers, the tools they use, and study them in detail. …

We have identified many different versions of IRC-based bots … The bot joins a specific IRC channel on an IRC server and waits there for further commands. This allows an attacker to remotely control this bot and use it for fun and also for profit. Attackers even go a step further and bring different bots together. Such a structure, consisting of many compromised machines which can be managed from an IRC channel, is called a botnet. IRC is not the best solution since the communication between bots and their controllers is rather bloated, a simpler communication protocol would suffice. But IRC offers several advantages: IRC Servers are freely available and are easy to set up, and many attackers have years of IRC communication experience.

… Even a relatively small botnet with only 1000 bots can cause a great deal of damage. These 1000 bots have a combined bandwidth (1000 home PCs with an average upstream of 128KBit/s can offer more than 100MBit/s) that is probably higher than the Internet connection of most corporate systems. In addition, the IP distribution of the bots makes ingress filter construction, maintenance, and deployment difficult. In addition, incident response is hampered by the large number of separate organizations involved. Another use for botnets is stealing sensitive information or identity theft: Searching some thousands home PCs for password.txt, or sniffing their traffic, can be effective.

The spreading mechanisms used by bots is a leading cause for “background noise” on the Internet, especially on TCP ports 445 and 135. … These malware scan large network ranges for new vulnerable computers and infect them, thus acting similar to a worm or virus. … most traffic targets the ports used for resource sharing on machines running all versions of Microsoft’s Windows operating system …

The traffic on these four ports [445/TCP, 139/TCP, 137/UDP, 135/TCP] cause more then 80 percent of the whole traffic captured. …

Lessons Learned

  • Number of botnets

    … able to track little more than 100 botnets during the last four months. … at the moment we are tracking about 35 active botnets.

  • Number of hosts

    During these few months, we saw 226,585 unique IP addresses joining at least one of the channels we monitored. … If an IRCd is modified not to show joining clients in a channel, we don’t see IPs here. Furthermore some IRCds obfuscate the joining clients IP address and obfuscated IP addresses do not count as seen, too. … this would mean that more then one million hosts are compromised and can be controlled by malicious attackers.

  • Typical size of Botnets

    Some botnets consist of only a few hundred bots. In contrast to this, we have also monitored several large botnets with up to 50.000 hosts. … botnets with over several hundred thousands hosts have been reported in the past. … We know about a home computer which got infected by 16 (sic!) different bots, so its hard to make an estimation about world bot population here.

  • Dimension of DDoS-attacks

    From the beginning of November 2004 until the end of January 2005, we were able to observe 226 DDoS-attacks against 99 unique targets.

  • Spreading of botnets

    “.advscan lsass 150 5 0 -r -s” and other commands are the most frequent observed messages. Through this and similar commands, bots spread and search for vulnerable systems.

  • Harvesting of information

    … harvesting of information from all compromised machines. With the help of a command like “.getcdkeys” the operator of the botnet is able to request a list of CD-keys (e.g. for Windows or games) from all bots.

  • “Updates” within botnets

    … observed updates of botnets quite frequently. … bots are instructed to download a piece of software from the Internet and then execute it. … bots can be dynamically updated and be further enhanced. … In total, we have collected 329 binaries. … Most of the other binary files are either adware …, proxy servers … or Browser Helper Objects.

An analysis of botnets Read More »

What is serious news reporting?

From Tom Stites’s “Guest Posting: Is Media Performance Democracy’s Critical Issue?” (Center for Citizen Media: Blog: 3 July 2006):

Serious reporting is based in verified fact passed through mature professional judgment. It has integrity. It engages readers – there’s that word again, readers – with compelling stories and it appeals to their human capacity for reason. This is the information that people need so they can make good life decisions and good citizenship decisions. Serious reporting is far from grim and solemn and off-putting. It is accessible and relevant to its readers. And the best serious reporting is a joy to read.

Serious reporting emanates largely from responsible local dailies and national and foreign reporting by big news organizations, print and broadcast. But the reporting all these institutions do is diminishing. With fewer reporters chasing the news, there is less and less variety in the stories citizens see and hear. The media that are booming, especially cable news and blogs, do precious little serious reporting. Or they do it for specialized audiences.

What is serious news reporting? Read More »

AACS, next-gen encryption for DVDs

From Nate Anderson’s “Hacking Digital Rights Management” (Ars Technica: 18 July 2006):

AACS relies on the well-established AES (with 128-bit keys) to safeguard the disc data. Just like DVD players, HD DVD and Blu-ray drives will come with a set of Device Keys handed out to the manufacturers by AACS LA. Unlike the CSS encryption used in DVDs, though, AACS has a built-in method for revoking sets of keys that are cracked and made public. AACS-encrypted discs will feature a Media Key Block that all players need to access in order to get the key needed to decrypt the video files on the disc. The MKB can be updated by AACS LA to prevent certain sets of Device Keys from functioning with future titles – a feature that AACS dubs “revocation.” …

AACS also supports a new feature called the Image Constraint Token. When set, the ICT will force video output to be degraded over analog connections. ICT has so far gone unused, though this could change at any time. …

While AACS is used by both HD disc formats, the Blu-ray Disc Association (BDA) has added some features of its own to make the format “more secure” than HD DVD. The additions are BD+ and ROM Mark; though both are designed to thwart pirates, they work quite differently.

While the generic AACS spec includes key revocation, BD+ actually allows the BDA to update the entire encryption system once players have already shipped. Should encryption be cracked, new discs will include information that will alter the players’ decryption code. …

The other new technology, ROM Mark, affects the manufacturing of Blu-ray discs. All Blu-ray mastering equipment must be licensed by the BDA, and they will ensure that all of it carries ROM Mark technology. Whenever a legitimate disc is created, it is given a “unique and undetectable identifier.” It’s not undetectable to the player, though, and players can refuse to play discs without a ROM Mark. The BDA has the optimistic hope that this will keep industrial-scale piracy at bay. We’ll see.

AACS, next-gen encryption for DVDs Read More »

To combat phishing, change browser design philosophy

From Federico Biancuzzi’s “Phishing with Rachna Dhamija” (SecurityFocus: 19 June 2006):

We discovered that existing security cues are ineffective, for three reasons:

1. The indicators are ignored (23% of participants in our study did not look at the address bar, status bar, or any SSL indicators).

2. The indicators are misunderstood. For example, one regular Firefox user told me that he thought the yellow background in the address bar was an aesthetic design choice of the website designer (he didn’t realize that it was a security signal presented by the browser). Other users thought the SSL lock icon indicated whether a website could set cookies.

3. The security indicators are trivial to spoof. Many users can’t distinguish between an actual SSL indicator in the browser frame and a spoofed image of that indicator that appears in the content of a webpage. For example, if you display a popup window with no address bar, and then add an image of an address bar at the top with the correct URL and SSL indicators and an image of the status bar at the bottom with all the right indicators, most users will think it is legitimate. This attack fooled more than 80% of participants. …

Currently, I’m working on other techniques to prevent phishing in conjunction with security skins. For example, in a security usability class I taught this semester at Harvard, we conducted a usability study that shows that simply showing a user’s history information (for example, “you’ve been to this website many times” or “you’ve never submitted this form before”) can significantly increase a user’s ability to detect a spoofed website and reduce their vulnerability to phishing attacks. Another area I’ve been investigating are techniques to help users recover from errors and to identify when errors are real, or when they are simulated. Many attacks rely on users not being able to make this distinction.

You presented the project called Dynamic Security Skins (DSS) nearly one year ago. Do you think the main idea behind it is still valid after your tests?

Rachna Dhamija: I think that our usability study shows how easy it is to spoof security indicators, and how hard it is for users to distinguish legitimate security indicators from those that have been spoofed. Dynamic Security Skins is a proposal that starts from the assumption that any static security indicator can easily be copied by attacker. Instead, we propose that users create their own customized security indicators that are hard for an attacker to predict. Our usability study also shows that indicators placed in the periphery or outside of the user’s focus of attention (such as the SSL lock icon in the status bar) may be ignored entirely by some users. DSS places the security indicator (a secret image) at the point of password entry, so the user can not ignore it.

DSS adds a trusted window in the browser dedicated to username and password entry. The user chooses a photographic image (or is assigned a random image), which is overlaid across the window and text entry boxes. If the window displays the user’s personal image, it is safe for the user to enter his password. …

With security skins, we were trying to solve not user authentication, but the reverse problem – server authentication. I was looking for a way to convey to a user that his client and the server had successfully negotiated a protocol, that they have mutually authenticated each other and agreed on the same key. One way to do this would be to display a message like “Server X is authenticated”, or to display a binary indicator, like a closed or open lock. The problem is that any static indicator can be easily copied by an attacker. Instead, we allow the server and the user’s browser to each generate an abstract image. If the authentication is successful, the two images will match. This image can change with each authentication. If it is captured, it can’t be replayed by an attacker and it won’t reveal anything useful about the user’s password. …

Instead of blaming specific development techniques, I think we need to change our design philosophy. We should assume that every interface we develop will be spoofed. The only thing an attacker can’t simulate is an interface he can’t predict. This is the principle that DSS relies on. We should make it easy for users to personalize their interfaces. Look at how popular screensavers, ringtones, and application skins are – users clearly enjoy the ability to personalize their interfaces. We can take advantage of this fact to build spoof resistant interfaces.

To combat phishing, change browser design philosophy Read More »

Differences between Macintosh & Unix programmers

From Eric Steven Raymond’s “Problems in the Environment of Unix” (The Art of Unix Programming: 19 September 2003):

Macintosh programmers are all about the user experience. They’re architects and decorators. They design from the outside in, asking first “What kind of interaction do we want to support?” and then building the application logic behind it to meet the demands of the user-interface design. This leads to programs that are very pretty and infrastructure that is weak and rickety. In one notorious example, as late as Release 9 the MacOS memory manager sometimes required the user to manually deallocate memory by manually chucking out exited but still-resident programs. Unix people are viscerally revolted by this kind of mal-design; they don’t understand how Macintosh people could live with it.

By contrast, Unix people are all about infrastructure. We are plumbers and stonemasons. We design from the inside out, building mighty engines to solve abstractly defined problems (like “How do we get reliable packet-stream delivery from point A to point B over unreliable hardware and links?”). We then wrap thin and often profoundly ugly interfaces around the engines. The commands date(1), find(1), and ed(1) are notorious examples, but there are hundreds of others. Macintosh people are viscerally revolted by this kind of mal-design; they don’t understand how Unix people can live with it. …

In many ways this kind of parochialism has served us well. We are the keepers of the Internet and the World Wide Web. Our software and our traditions dominate serious computing, the applications where 24/7 reliability and minimal downtime is a must. We really are extremely good at building solid infrastructure; not perfect by any means, but there is no other software technical culture that has anywhere close to our track record, and it is one to be proud of. …

To non-technical end users, the software we build tends to be either bewildering and incomprehensible, or clumsy and condescending, or both at the same time. Even when we try to do the user-friendliness thing as earnestly as possible, we’re woefully inconsistent at it. Many of the attitudes and reflexes we’ve inherited from old-school Unix are just wrong for the job. Even when we want to listen to and help Aunt Tillie, we don’t know how — we project our categories and our concerns onto her and give her ‘solutions’ that she finds as daunting as her problems.

Differences between Macintosh & Unix programmers Read More »

The real purposes of the American school

From John Taylor Gatto’s “Against School” (Harper’s Magazine: September 2003):

Mass schooling of a compulsory nature really got its teeth into the United States between 1905 and 1915, though it was conceived of much earlier and pushed for throughout most of the nineteenth century. The reason given for this enormous upheaval of family life and cultural traditions was, roughly speaking, threefold:

1) To make good people.
2) To make good citizens.
3) To make each person his or her personal best.

These goals are still trotted out today on a regular basis, and most of us accept them in one form or another as a decent definition of public education’s mission, however short schools actually fall in achieving them. But we are dead wrong. Compounding our error is the fact that the national literature holds numerous and surprisingly consistent statements of compulsory schooling’s true purpose. We have, for example, the great H. L. Mencken, who wrote in The American Mercury for April 1924 that the aim of public education is not

to fill the young of the species with knowledge and awaken their intelligence. . . . Nothing could be further from the truth. The aim.. . is simply to reduce as many individuals as possible to the same safe level, to breed and train a standardized citizenry, to put down dissent and originality. That is its aim in the United States . . . and that is its aim everywhere else.

[Alexander Inglis, author of the 1918 book, Principles of Secondary Education,], for whom a lecture in education at Harvard is named, makes it perfectly clear that compulsory schooling on this continent was intended to be just what it had been for Prussia in the 1820s: a fifth column into the burgeoning democratic movement that threatened to give the peasants and the proletarians a voice at the bargaining table. Modern, industrialized, compulsory schooling was to make a sort of surgical incision into the prospective unity of these underclasses. Divide children by subject, by age-grading, by constant rankings on tests, and by many other more subtle means, and it was unlikely that the ignorant mass of mankind, separated in childhood, would ever reintegrate into a dangerous whole.

Inglis breaks down the purpose – the actual purpose – of modem schooling into six basic functions, any one of which is enough to curl the hair of those innocent enough to believe the three traditional goals listed earlier:

1) The adjustive or adaptive function. Schools are to establish fixed habits of reaction to authority. This, of course, precludes critical judgment completely. It also pretty much destroys the idea that useful or interesting material should be taught, because you can’t test for reflexive obedience until you know whether you can make kids learn, and do, foolish and boring things.

2) The integrating function. This might well be called “the conformity function,” because its intention is to make children as alike as possible. People who conform are predictable, and this is of great use to those who wish to harness and manipulate a large labor force.

3) The diagnostic and directive function. School is meant to determine each student’s proper social role. This is done by logging evidence mathematically and anecdotally on cumulative records. As in “your permanent record.” Yes, you do have one.

4) The differentiating function. Once their social role has been “diagnosed,” children are to be sorted by role and trained only so far as their destination in the social machine merits – and not one step further. So much for making kids their personal best.

5) The selective function. This refers not to human choice at all but to Darwin’s theory of natural selection as applied to what he called “the favored races.” In short, the idea is to help things along by consciously attempting to improve the breeding stock. Schools are meant to tag the unfit – with poor grades, remedial placement, and other punishments – clearly enough that their peers will accept them as inferior and effectively bar them from the reproductive sweepstakes. That’s what all those little humiliations from first grade onward were intended to do: wash the dirt down the drain.

6) The propaedeutic function. The societal system implied by these rules will require an elite group of caretakers. To that end, a small fraction of the kids will quietly be taught how to manage this continuing project, how to watch over and control a population deliberately dumbed down and declawed in order that government might proceed unchallenged and corporations might never want for obedient labor. …

Class may frame the proposition, as when Woodrow Wilson, then president of Princeton University, said the following to the New York City School Teachers Association in 1909: “We want one class of persons to have a liberal education, and we want another class of persons, a very much larger class, of necessity, in every society, to forgo the privileges of a liberal education and fit themselves to perform specific difficult manual tasks.” …

Now, you needn’t have studied marketing to know that there are two groups of people who can always be convinced to consume more than they need to: addicts and children. School has done a pretty good job of turning our children into addicts, but it has done a spectacular job of turning our children into children. Again, this is no accident. Theorists from Plato to Rousseau to our own Dr. Inglis knew that if children could be cloistered with other children, stripped of responsibility and independence, encouraged to develop only the trivializing emotions of greed, envy, jealousy, and fear, they would grow older but never truly grow up. …

Now for the good news. Once you understand the logic behind modern schooling, its tricks and traps are fairly easy to avoid. School trains children to be employees and consumers; teach your own to be leaders and adventurers. School trains children to obey reflexively; teach your own to think critically and independently. Well-schooled kids have a low threshold for boredom; help your own to develop an inner life so that they’ll never be bored. Urge them to take on the serious material, the grown-up material, in history, literature, philosophy, music, art, economics, theology – all the stuff schoolteachers know well enough to avoid. Challenge your kids with plenty of solitude so that they can learn to enjoy their own company, to conduct inner dialogues. Well-schooled people are conditioned to dread being alone, and they seek constant companionship through the TV, the computer, the cell phone, and through shallow friendships quickly acquired and quickly abandoned. Your children should have a more meaningful life, and they can.

First, though, we must wake up to what our schools really are: laboratories of experimentation on young minds, drill centers for the habits and attitudes that corporate society demands. Mandatory education serves children only incidentally; its real purpose is to turn them into servants. Don’t let your own have their childhoods extended, not even for a day. If David Farragut could take command of a captured British warship as a preteen, if Thomas Edison could publish a broadsheet at the age of twelve, if Ben Franklin could apprentice himself to a printer at the same age (then put himself through a course of study that would choke a Yale senior today), there’s no telling what your own kids could do. After a long life, and thirty years in the public school trenches, I’ve concluded that genius is as common as dirt. We suppress our genius only because we haven’t yet figured out how to manage a population of educated men and women. The solution, I think, is simple and glorious. Let them manage themselves.

The real purposes of the American school Read More »

The birth of Geology & gradualism as a paradigm shift from catastrophism

From Kim Stanley Robinson’s “Imagining Abrupt Climate Change : Terraforming Earth” (Amazon Shorts: 31 July 2005):

This view, by the way, was in keeping with a larger and older paradigm called gradualism, the result of a dramatic and controversial paradigm shift of its own from the nineteenth century, one that is still a contested part of our culture wars, having to do with the birth of geology as a field, and its discovery of the immense age of the Earth. Before that, Earth’s history tended to be explained in a kind of Biblical paradigm, in which the Earth was understood to be several thousand years old, because of genealogies in the Bible, so that landscape features tended to be explained by events like Noah’s flood. This kind of “catastrophism” paradigm was what led Josiah Whitney to maintain that Yosemite Valley must have been formed by a cataclysmic earthquake, for instance; there simply hadn’t been time for water and ice to have carved something as hard as granite. It was John Muir who made the gradualist argument for glacial action over millions of years; and the eventual acceptance of his explanation was part of the general shift to gradualist explanations for Earth’s landforms, which also meant there was another time for evolution to have taken place. Gradualism also led by extension to thinking that the various climate regimes of the past had also come about fairly gradually.

The birth of Geology & gradualism as a paradigm shift from catastrophism Read More »

Why structureless is not only impossible, but counterproductive

From Jo Freeman’s “The Tyranny of Structurelessness” (1970):

During the years in which the women’s liberation movement has been taking shape, a great emphasis has been placed on what are called leaderless, structureless groups as the main form of the movement. …

The idea of ‘structurelessness’, however, has moved from a healthy counter to these tendencies to becoming a goddess in its own right. The idea is as little examined as the term is much used, but it has become an intrinsic and unquestioned part of women’s liberation ideology. …

If the movement is to move beyond these elementary stages of development, it will have to disabuse itself of some of its prejudices about organisation and structure. There is nothing inherently bad about either of these. …

Contrary to what we would like to believe, there is no such thing as a ‘structureless’ group. Any group of people of whatever nature coming together for any length of time, for any purpose, will inevitably structure itself in some fashion. The structure may be flexible, it may vary over time, it may evenly or unevenly distribute tasks, power and resources over the members of the group. But it will be formed regardless of the abilities, personalities and intentions of the people involved. The very fact that we are individuals with different talents, predisposition’s and backgrounds makes this inevitable. Only if we refused to relate or interact on any basis whatsoever could we approximate ‘structurelessness’ and that is not the nature of a human group. …

Thus ‘structurelessness’ becomes a way of masking power, and within the women’s movement it is usually most strongly advocated by those who are the most powerful (whether they are conscious of their power or not). The rules of how decisions are made are known only to a few and awareness of power is curtailed by those who know the rules, as long as the structure of the group is informal. Those who do not know the rules and are not chosen for initiation must remain in confusion, or suffer from paranoid delusions that something is happening of which they are not quite aware. …

A structured group always has a formal structure, and may also have an informal one. An unstructured group always has an informal , or covert, structure. It is this informal structure, particularly in unstructured groups, which forms the basis for elites. …

Correctly, an elite refers to a small group of people who have power over a larger group of which they are part, usually without direct responsibility to that larger group, and often without their knowledge or consent. A person becomes an elitist by being part of, or advocating, the rule by such a small group, whether or not that individual is well-known or not known at all. Notoriety is not a definition of an elitist. The most insidious elites are usually run by people not known to the larger public at all. Intelligent elitists are usually smart enough not to allow themselves to become well- known. When they become known, they are watched, and the mask over their power is no longer firmly lodged. …

Only three techniques have ever been developed for establishing mass group opinion: the vote or referendum, the public opinion survey questionnaire and the selection of group spokespeople at an appropriate meeting. The women’s liberation movement has used none of these to communicate with the public. Neither the movement as a whole nor most of the multitudinous groups within it have established a means of explaining their position on various issues. But the public is conditioned to look for spokespeople. …

The more unstructured a movement is, the less control it has over the directions in which it develops and the political actions in which it engages. This does not mean that its ideas do not spread. Given a certain amount of interest by the media and the appropriateness of social conditions, the ideas will still be diffused widely. But diffusion of ideas does not mean they are implemented; it only means they are talked about. Insofar as they can be applied individually they may be acted upon; insofar as they require co-ordinated political power to be implemented, they will not be.

Why structureless is not only impossible, but counterproductive Read More »

The differences between language in art & politics

From Harold Pinter’s “Nobel Lecture: Art, Truth & Politics” (Nobel Prize: 7 December 2005):

In 1958 I wrote the following:

‘There are no hard distinctions between what is real and what is unreal, nor between what is true and what is false. A thing is not necessarily either true or false; it can be both true and false.’

I believe that these assertions still make sense and do still apply to the exploration of reality through art. So as a writer I stand by them but as a citizen I cannot. As a citizen I must ask: What is true? What is false? …

So language in art remains a highly ambiguous transaction, a quicksand, a trampoline, a frozen pool which might give way under you, the author, at any time. …

Political language, as used by politicians, does not venture into any of this territory since the majority of politicians, on the evidence available to us, are interested not in truth but in power and in the maintenance of that power. To maintain that power it is essential that people remain in ignorance, that they live in ignorance of the truth, even the truth of their own lives. What surrounds us therefore is a vast tapestry of lies, upon which we feed.

The differences between language in art & politics Read More »

Why software is difficult to create … & will always be difficult

From Frederick P. Brooks, Jr.’s “No Silver Bullet: Essence and Accidents of Software Engineering” (Computer: Vol. 20, No. 4 [April 1987] pp. 10-19):

The familiar software project, at least as seen by the nontechnical manager, has something of this character; it is usually innocent and straightforward, but is capable of becoming a monster of missed schedules, blown budgets, and flawed products. So we hear desperate cries for a silver bullet–something to make software costs drop as rapidly as computer hardware costs do.

But, as we look to the horizon of a decade hence, we see no silver bullet. There is no single development, in either technology or in management technique, that by itself promises even one order-of-magnitude improvement in productivity, in reliability, in simplicity. …

The essence of a software entity is a construct of interlocking concepts: data sets, relationships among data items, algorithms, and invocations of functions. This essence is abstract in that such a conceptual construct is the same under many different representations. It is nonetheless highly precise and richly detailed.

I believe the hard part of building software to be the specification, design, and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation. We still make syntax errors, to be sure; but they are fuzz compared with the conceptual errors in most systems. …

Let us consider the inherent properties of this irreducible essence of modern software systems: complexity, conformity, changeability, and invisibility.

Complexity. Software entities are more complex for their size than perhaps any other human construct because no two parts are alike (at least above the statement level). …

Many of the classic problems of developing software products derive from this essential complexity and its nonlinear increases with size. From the complexity comes the difficulty of communication among team members, which leads to product flaws, cost overruns, schedule delays. From the complexity comes the difficulty of enumerating, much less understanding, all the possible states of the program, and from that comes the unreliability. From complexity of function comes the difficulty of invoking function, which makes programs hard to use. From complexity of structure comes the difficulty of extending programs to new functions without creating side effects. From complexity of structure come the unvisualized states that constitute security trapdoors.

Not only technical problems, but management problems as well come from the complexity. It makes overview hard, thus impeding conceptual integrity. It makes it hard to find and control all the loose ends. It creates the tremendous learning and understanding burden that makes personnel turnover a disaster.

Conformity. … No such faith comforts the software engineer. Much of the complexity that he must master is arbitrary complexity, forced without rhyme or reason by the many human institutions and systems to which his interfaces must conform. …

Changeability. … All successful software gets changed. Two processes are at work. First, as a software product is found to be useful, people try it in new cases at the edge of or beyond the original domain. The pressures for extended function come chiefly from users who like the basic function and invent new uses for it.

Second, successful software survives beyond the normal life of the machine vehicle for which it is first written. If not new computers, then at least new disks, new displays, new printers come along; and the software must be conformed to its new vehicles of opportunity. …

Invisibility. Software is invisible and unvisualizable. …

The reality of software is not inherently embedded in space. Hence, it has no ready geometric representation in the way that land has maps, silicon chips have diagrams, computers have connectivity schematics. As soon as we attempt to diagram software structure, we find it to constitute not one, but several, general directed graphs superimposed one upon another. The several graphs may represent the flow of control, the flow of data, patterns of dependency, time sequence, name-space relationships. These graphs are usually not even planar, much less hierarchical. …

Past Breakthroughs Solved Accidental Difficulties

If we examine the three steps in software technology development that have been most fruitful in the past, we discover that each attacked a different major difficulty in building software, but that those difficulties have been accidental, not essential, difficulties. …

High-level languages. Surely the most powerful stroke for software productivity, reliability, and simplicity has been the progressive use of high-level languages for programming. …

What does a high-level language accomplish? It frees a program from much of its accidental complexity. …

Time-sharing. Time-sharing brought a major improvement in the productivity of programmers and in the quality of their product, although not so large as that brought by high-level languages.

Time-sharing attacks a quite different difficulty. Time-sharing preserves immediacy, and hence enables one to maintain an overview of complexity. …

Unified programming environments. Unix and Interlisp, the first integrated programming environments to come into widespread use, seem to have improved productivity by integral factors. Why?

They attack the accidental difficulties that result from using individual programs together, by providing integrated libraries, unified file formats, and pipes and filters. As a result, conceptual structures that in principle could always call, feed, and use one another can indeed easily do so in practice.

Why software is difficult to create … & will always be difficult Read More »

Paradigm shifts explained

From Kim Stanley Robinson’s “Imagining Abrupt Climate Change : Terraforming Earth” (Amazon Shorts: 31 July 2005):

… paradigm shifts are exciting moments in science’s ongoing project of self-improvement, making itself more accurately mapped to reality as it is discovered and teased out; this process of continual recalibration and improvement is one of the most admirable parts of science, which among other things is a most powerful and utopian set of mental habits; an attitude toward reality that I have no hesitation in labeling a kind of worship or devotion. And in this ongoing communal act of devotion, paradigm shifts are very good at revealing how science is conducted, in part because each one represents a little (or big) crisis of understanding.

As Thomas Kuhn described the process in his seminal book The Structure of Scientific Revolutions, workers in the various branches of science build over time an interconnected construct of concepts and beliefs that allow them to interpret the data from their experiments, and fit them into a larger picture of the world that makes the best sense of the evidence at hand. What is hoped for is a picture that, if anyone else were to question it, and follow the train of reasoning and all the evidence used to support it, they too would agree with it. This is one of the ways science is interestingly utopian; it attempts to say things that everyone looking at the same evidence would agree to.

So, using this paradigm, always admitted to be a work in progress, scientists then conduct what Kuhn calls “normal science,” elucidating further aspects of reality by using the paradigm to structure their questions and their answers. Sometimes paradigms are useful for centuries; other times, for shorter periods. Then it often happens that scientists in the course of doing “normal science” begin to get evidence from the field that cannot be explained within the paradigm that has been established. At first such “anomalies” are regarded as suspect in themselves, precisely because they don’t fit the paradigm. They’re oddities, and something might be wrong with them as such. Thus they are ignored, or tossed aside, or viewed with suspicion, or in some other way bracketed off. Eventually, if enough of them pile up, and they seem similar in kind, or otherwise solid as observations, attempts might be made to explain them within the old paradigm, by tweaking or re-interpreting the paradigm itself, without actually throwing the paradigm out entirely.

For instance, when it was found that Newtonian laws of gravitation could not account for the speed of Mercury, which was moving a tiny bit faster than it ought to have been, even though Newton’s laws accounted for all the other planets extremely well, at first some astronomers suggested there might be another planet inside the orbit of Mercury, too close to the Sun for us to see. They even gave this potential planet a name, Vulcan; but they couldn’t see it, and calculations revealed that this hypothetical Vulcan still would not explain the discrepancy in Mercury’s motion. The discrepancy remained an anomaly, and was real enough and serious enough to cast the whole Newtonian paradigm into doubt among the small group of people who worried about it and wondered what could be causing it.

It was Einstein who then proposed that Mercury moved differently than predicted because spacetime itself curved around masses, and near the huge mass of the Sun the effect was large enough to be noticeable.

Whoah! This was a rather mind-bogglingly profound explanation for a little orbital discrepancy in Mercury; but Einstein also made a new prediction and suggested an experiment; if his explanation were correct, then light too would bend in the gravity well around the sun, and so the light of a star would appear from behind the sun a little bit before the astronomical tables said that it should. The proposed experiment presented some observational difficulties, but a few years later it was accomplished during a total eclipse of the sun, and the light of a certain star appeared before it ought to have by just the degree Einstein had predicted. And so Einstein’s concepts concerning spacetime began to be accepted and elaborated, eventually forming a big part of the paradigm known as the “standard model,” within which new kinds of “normal science” in physics and astronomy could be done. …

Paradigm shifts explained Read More »

How doctors measure what percentage of your body is burned

From Daniel Engber’s “How Much of Me Is Burned?” (Slate: 11 July 2006):

rule-of-nines.gif In the 1950s, doctors developed an easy way to estimate the ratio of the area of a patient’s burns to the total area of his skin. The system works by assigning standard percentages to major body parts. (Most of these happen to be multiples of nine.) The skin on each arm, for example, covers 9 percent of a patient’s total surface area. Each leg comprises 18 percent, as do the front and back of the torso. The head and neck together make up another 9 percent, and the last bit (or 1 percent) covers the genitalia and perineum. This breakdown makes it easy for doctors to estimate the size of a burn in relation to a body—a burn that covered half the arm would add 4 or 5 percent to the total figure. …

Another method uses the size of a patient’s palm as a reference. As a general rule, the skin on the palm of your hand comprises 0.5 percent of your total surface area. (For children, it’s 1 percent.) A doctor can check the size of a patient’s hand and compare it with the size of a burn to make a quick guess about the percentage.

How doctors measure what percentage of your body is burned Read More »

Why no terrorist attacks since 9/11?

From Bruce Schneier’s “Movie Plot Threat Contest: Status Report” (Crypto-Gram Newsletter: 15 May 2006):

… you have to wonder why there have been no terrorist attacks in the U.S. since 9/11. I don’t believe the “flypaper theory” that the terrorists are all in Iraq instead of in the U.S. And despite all the ineffectual security we’ve put in place since 9/11, I’m sure we have had some successes in intelligence and investigation — and have made it harder for terrorists to operate both in the U.S. and abroad.

But mostly, I think terrorist attacks are much harder than most of us think. It’s harder to find willing recruits than we think. It’s harder to coordinate plans. It’s harder to execute those plans. Terrorism is rare, and for all we’ve heard about 9/11 changing the world, it’s still rare.

Why no terrorist attacks since 9/11? Read More »

THE answer to “if you’re not doing anything wrong, why resist surveillance?”

From Bruce Schneier’s “The Eternal Value of Privacy” (Wired News: 18 May 2006):

The most common retort against privacy advocates — by those in favor of ID checks, cameras, databases, data mining and other wholesale surveillance measures — is this line: “If you aren’t doing anything wrong, what do you have to hide?”

Some clever answers: “If I’m not doing anything wrong, then you have no cause to watch me.” “Because the government gets to define what’s wrong, and they keep changing the definition.” “Because you might do something wrong with my information.” My problem with quips like these — as right as they are — is that they accept the premise that privacy is about hiding a wrong. It’s not. Privacy is an inherent human right, and a requirement for maintaining the human condition with dignity and respect.

Two proverbs say it best: Quis custodiet custodes ipsos? (“Who watches the watchers?”) and “Absolute power corrupts absolutely.”

Cardinal Richelieu understood the value of surveillance when he famously said, “If one would give me six lines written by the hand of the most honest man, I would find something in them to have him hanged.” Watch someone long enough, and you’ll find something to arrest — or just blackmail — with. Privacy is important because without it, surveillance information will be abused: to peep, to sell to marketers and to spy on political enemies — whoever they happen to be at the time.

THE answer to “if you’re not doing anything wrong, why resist surveillance?” Read More »

Unix specs vs. Windows specs

From Peter Seebach’s Standards and specs: Not by UNIX alone (IBM developerWorks: 8 March 2006):

In the past 20 years, developers for “the same” desktop platform (“whatever Microsoft ships”) have been told that the API to target is (in this order):

* DOS
* Win16
* OS/2
* Win32
* WinNT
* WinXP
* and most recently .NET.

Of course, that list is from last year, and now the “stable” target that you should be developing for, if you have an eye for the future, is Vista.

It hasn’t been quite as bad in the Macintosh world, where the number of major API changes has been limited: classic single-tasking Mac OS, classic multitasking Mac OS (System 7), Carbon (System 8/9 and preview of OS X), and Cocoa (OS X), but even there, the cost of migration has been significant. At least OS X finally offers a stable UNIX API for the back-end part of programs, allowing developers to ignore the API creep except in GUI code.

By contrast, twenty-year-old UNIX utilities still compile and run. A new desktop computing API will come and everyone will have to rewrite for it, but mountains will erode away before read() and write() stop working. This is the reason that all the hassle of formal UNIX standards has had so little effect on practical UNIX software development; the core API is simple, clean, and well-designed, and there is no need to change it significantly.

… UNIX users have been switching hardware platforms since the 1970s; it’s no big deal. …

Just as there are many varieties of UNIX, there are many UNIX standards:

* Probably the oldest standard that people still refer to is AT&T’s 1985 System V Interface Definition (SVID). This standard shows up, for instance, in man pages describing the standards compliance of functions that have been in the C library “forever.”
* Meanwhile, X/Open (now the Open Group) was developing “portability guides” with names like XPG2, XPG3, and so on. XPG1 was actually released in 1995. The XPG guides are largely subsumed into newer specs, but once again, are still referred to sometimes in documentation.
* The IEEE’s POSIX standard showed up in 1990 with updates in 1992 and 1993 and a second edition in 1996. It’s still a viable standard, although it has suffered from poor accessibility. POSIX specs have names like 1003.x; for instance, 1003.1 and 1003.2, which refer to different parts of the standard, or 1003.1-1988 and 1003.1-1990, which refer to two versions of the standard.
* The fairly ominous sounding “Spec 1170” (also known as “UNIX 98” or “Single Unix Specification”) is probably the most complete specification; it is produced by the Open Group, and is effectively a descendant of the XPG series. In practice, this is “the” UNIX standard these days, although it’s a little large; this has had an impact on conformance testing.
* The Linux Standards Base is not strictly a UNIX standard, but it’s a standardization effort relevant to a very large number of developers working with code designed to run “on UNIX.” …

You can look at OS specifications in two very different ways: one is from the point of view of a developer trying to port an application, and the other is from the point of view of the user trying to interact with the system.

UNIX conveniently blurs this distinction. The primary user interface is also one of the primary development environments; therefore, UNIX specifications often cover not only the C language API, but also the shell environment and many of the core utilities shell programmers rely on. …

From the perspective of a developer who’s seen many Unix-like systems, Linux is probably mostly sort of similar to System V. The heavy focus on GNU utilities gives a sort of surreal combination of Berkeley and System V features, but if you have to guess whether Linux does something the Berkeley way or the System V way, go with System V. This is especially true of system startup; nearly all Linux systems use the System V /etc/inittab and /etc/rc.d structure, or something very close to it. …

Unix specs vs. Windows specs Read More »

DIY worm kits

From Jose Nazario’s Anatomy of a worm (Computerworld: 15 September 2004):

Now imagine a world where worm attacks frequently occur because hackers and rogue developers have access to “worm kits” or development tools that provide the basic building blocks for rapid worm development.

Historically, worms were basic clones of one another that didn’t change after their original development. Simple mechanisms were used to propagate them, such as mass-mailing worms using a single subject line.

Today’s worms are more sophisticated. They have the ability to mutate after development based on knowledge of how to thwart new security processes. For instance, an early worm, Code Red, attacked only Internet Information Server servers. The Nimda worm, which came later, expanded to include at least three additional attack methodologies: mail-based attacks, file-sharing-based attacks, and attacks against the Internet Explorer Web browser.

The potential for this worm-a-day nightmare comes from several factors: the dozens of vulnerabilities that are ready to be exploited, the availability of worm source code, recycled exploits and the ease of editing existing worms.

DIY worm kits Read More »

Remote fingerprinting of devices connected to the Net

Anonymous Internet access is now a thing of the past. A doctoral student at the University of California has conclusively fingerprinted computer hardware remotely, allowing it to be tracked wherever it is on the Internet.

In a paper on his research, primary author and Ph.D. student Tadayoshi Kohno said: “There are now a number of powerful techniques for remote operating system fingerprinting, that is, remotely determining the operating systems of devices on the Internet. We push this idea further and introduce the notion of remote physical device fingerprinting … without the fingerprinted device’s known cooperation.”

The potential applications for Kohno’s technique are impressive. For example, “tracking, with some probability, a physical device as it connects to the Internet from different access points, counting the number of devices behind a NAT even when the devices use constant or random IP identifications, remotely probing a block of addresses to determine if the addresses correspond to virtual hosts (for example, as part of a virtual honeynet), and unanonymising anonymised network traces.” …

Another application for Kohno’s technique is to “obtain information about whether two devices on the Internet, possibly shifted in time or IP addresses, are actually the same physical device.”

The technique works by “exploiting small, microscopic deviations in device hardware: clock skews.” In practice, Kohno’s paper says, his techniques “exploit the fact that most modern TCP stacks implement the TCP timestamps option from RFC 1323 whereby, for performance purposes, each party in a TCP flow includes information about its perception of time in each outgoing packet. A fingerprinter can use the information contained within the TCP headers to estimate a device’s clock skew and thereby fingerprint a physical device.”

Kohno goes on to say: ” Our techniques report consistent measurements when the measurer is thousands of miles, multiple hops, and tens of milliseconds away from the fingerprinted device, and when the fingerprinted device is connected to the Internet from different locations and via different access technologies. Further, one can apply our passive and semi-passive techniques when the fingerprinted device is behind a NAT or firewall.”

And the paper stresses that “For all our methods, we stress that the fingerprinter does not require any modification to or cooperation from the fingerprintee.” Kohno and his team tested their techniques on many operating systems, including Windows XP and 2000, Mac OS X Panther, Red Hat and Debian Linux, FreeBSD, OpenBSD and even Windows for Pocket PCs 2002.

Remote fingerprinting of devices connected to the Net Read More »

Windows directory services

From David HM Spector’s Unfinished Business Part 2: Closing the Circle (LinuxDevCenter: 7 July 2003):

… an integrated enterprise directory service does give network managers a much greater ability to manage large-scale networks and resources from almost every perspective.

Unlike most UNIX systems, Windows environments are homogeneous. There are three modes of operation in terms of user and resource management in the Windows universe:

1. Stand-alone.
2. Domain membership through a domain controller.
3. Organizational-unit membership in an LDAP-based directory such as Active Directory (or via a third-party directory such as NDS, but those are declining as more organizations switch to AD). …

Three major pieces of software make up the bulk of what Active Directory does:

* LDAP, the Lightweight Directory Access Protocol.
* Kerberos, the authorization system originally developed as part of MIT Athena (later, the basis for the security components in OSF’s DME).
* A SQL database.

These components interact with the Windows APIs to deliver a one-stop repository for any attribute that can be used to describe a system, a service, a device, users, groups, a relationship, a policy, an authorization, or another relationship in a computing environment. …

LDAP in AD is used to manage:

* DNS addresses
* Workstation and server descriptions
* Printers
* Print queues
* Volume mappings
* Certificates
* Licenses
* Policies (such as ACLs, security policies, etc.)
* Groups
* Users
* Contacts

All of these data are stored in one unified system, which can be broken down relatively easily (with some major caveats) by physical location (site), division, organization unit, or department and workgroup, and managed in a distributed fashion. These data can be replicated for redundancy and performance purposes. All Windows APIs must operate within this system if they are to participate in the network and have access to its resources. Repository data is wrapped up by and authenticated through the use of Kerberos Tickets, which makes the system (again, general Windows caveats applied) secure. …

The most interesting part of this story is that 95% of the hard work has already been done! Microsoft didn’t invent totally new LDAP schemas to make Active Directory as comprehensive as it is — as usual, they embraced and extended the work of others. LDAP schemas already exist, and are publicly available to cover:

* Contact management: The InetOrgPerson schema
* IP Addresses, Users, Server/Workstation Info: The NIS schema
* Kerberos tickets: IETF Kerberos KDC schema

Of course, Microsoft’s own schemas are available for perusal on any Active Directory server (or, if you happen to have a Macintosh OS X box, look in /etc/openldap, for all of Microsoft’s schemas are there). …

Windows directory services Read More »

Unix vs Windows: NYC vs Celebration

From David HM Spector’s Unfinished Business Part 2: Closing the Circle (LinuxDevCenter: 7 July 2003):

The UNIX world is the result of natural evolution, not the outgrowth of a planned community. UNIX is a lot like New York City: dynamic, always reinventing itself, adapting to new needs and realities. Windows is a lot like Celebration, USA: static, a set piece of predictability, slow to provide new services and very resistant to change or difference of view or opinion.

Unix vs Windows: NYC vs Celebration Read More »

How virtual machines work

From Samuel T. King, Peter M. Chen, Yi-Min Wang, Chad Verbowski, Helen J. Wang, & Jacob R. Lorch’s “SubVirt: Implementing malware with virtual machines
” [PDF] (: ):

A virtual-machine monitor (VMM) manages the resources of the underlying hardware and provides an abstraction of one or more virtual machines [20]. Each virtual machine can run a complete operating system and its applications. Figure 1 shows the architecture used by two modern VMMs (VMware and VirtualPC). Software running within a virtual machine is called guest software (i.e., guest operating systems and guest applications). All guest software (including the guest OS) runs in user mode; only the VMM runs in the most privileged level (kernel mode). The host OS in Figure 1 is used to provide portable access to a wide variety of I/O devices [44].

VMMs export hardware-level abstractions to guest software using emulated hardware. The guest OS interacts with the virtual hardware in the same manner as it would with real hardware (e.g., in/out instructions, DMA), and these interactions are trapped by the VMM and emulated in software. This emulation allows the guest OS to run without modification while maintaining control over the system at the VMM layer.

A VMM can support multiple OSes on one computer by multiplexing that computer’s hardware and providing the illusion of multiple, distinct virtual computers, each of which can run a separate operating system and its applications. The VMM isolates all resources of each virtual computer through redirection. For example, the VMM can map two virtual disks to different sectors of a shared physical disk, and the VMM can map the physical memory space of each virtual machine to different pages in the real machine’s memory. In addition to multiplexing a computer’s hardware, VMMs also provide a powerful platform for adding services to an existing system. For example, VMMs have been used to debug operating systems and system configurations [30, 49], migrate live machines [40], detect or prevent intrusions [18, 27, 8], and attest for code integrity [17]. These VM services are typically implemented outside the guest they are serving in order to avoid perturbing the guest.

One problem faced by VM services is the difficulty in understanding the states and events inside the guest they are serving; VM services operate at a different level of abstraction from guest software. Software running outside of a virtual machine views lowlevel virtual-machine state such as disk blocks, network packets, and memory. Software inside the virtual machine interprets this state as high-level abstractions such as files, TCP connections, and variables. This gap between the VMM’s view of data/events and guest software’s view of data/events is called the semantic gap [13].

Virtual-machine introspection (VMI) [18, 27] describes a family of techniques that enables a VM service to understand and modify states and events within the guest. VMI translates variables and guest memory addresses by reading the guest OS and applications’ symbol tables and page tables. VMI uses hardware or software breakpoints to enable a VM service to gain control at specific instruction addresses. Finally, VMI allows a VM service to invoke guest OS or application code. Invoking guest OS code allows the VM service to leverage existing, complex guest code to carry out general-purpose functionality such as reading a guest file from the file cache/disk system. VM services can protect themselves from guest code by disallowing external I/O. They can protect the guest data from perturbation by checkpointing it before changing its state and rolling the guest back later.

How virtual machines work Read More »