Classical design & evaluation
Design, Build, Use
- Document the concept and specify the requirements
- Modularize; design each part
- Code each component; integrate the pieces
- Test the system (verify)
- Use the system: Deploy and maintain

Problems with the waterfall model
- Synergies of components
- Problems hidden until full system t
- Information ecology differences
- Unknown user needs
- Races and users understanding of the functions
- User needs and situation change
- Non-standard use
Modifications
- Spiral design
- Overlapping phases
- Sub projects
- Evolutionary prototyping
- Stage delivery
- User-centered design
Reverse the flow?
Use, Build, Design
Pragmatism
- Technological change
- Scientific discoveries
- Demographics shifts
An idea about ideas
Technology signifies all the intelligent techniques by which the energies of nature and
man are directed and used in the satisfaction of human needs.John Dewey, What I Believe, 1930
Action -> -> -> Destiny
Plant a thought and reap a word;
Bishop Beckwaith, 1885
plant a word and reap an action;
plant an action and reap a habit;
plant a habit and reap a character;
plant a character and reap a destiny.
…habits are arts. They involve skill of sensory and motor organs, cunning or craft, and objective materials. They assimilate objective energies, and eventuate in command of environment. They require order, discipline, and manifest technique. They have a beginning, middle and end. Each stage marks progress in dealing with materials and tools, advance in converting material to active use.
John Dewey. Human Nature and Conduct, 1922
Pragmatist themes
• experimentalist philosophy • ordinary experience
• means vs. ends
• knowing — action
• relationships
• situation as a whole
• logic — inquiry into inquiry
• technology — means of resolving a problem
Pragmatic technology
technology as the means for resolving a problematic situation
— Larry Hickman (1990), John Dewey’s Pragmatic Technology
problem 1 => technology 1
technology 1 => problem 2
problem 2 => technology 2
technology 2 => problem 3
Problems find technologies
technology => solves a problem
solution to a problem => technology
Problem-solving cycle
problem 2 => technology 2 technology 2 => problem 3 …
Implications
• Technology studies: Alliance teams • Evaluation: ENFI
• Design: Inquiry Page
Case study: New ways of doing science in distributed teams
- Study of the Alliance/NCSA
How does embedded knowledge become mobile?

Application Technologies teams
Grand vision

Problems
- EOT often shows the greatest impact
- But it doesn’t use AT enough, and AT doesn’t use ET enough
- Successes often emerge from user community and are fed back into the Alliance
- Large structure w/o clear lines of control leads to politics, miscommunications, difficulty in planning, failures to collaborate effectively
Enabling Technologies teams
Astronomy Digital Imaging Library
Our vision is to create a cooperative system amongst astronomical researchers, professional societies, publishers, editors, libraries, and the Virtual Observatory in which the literature, the associated digital data, and the underlying data archives interoperate seamlessly and where high-level data products are assured of long-term preservation and curation.
Sayeed Choudhury, et al., Digital Data Preservation for Scholarly Publications in Astronomy, 2007
• developed and maintained by the Radio Astronomy Imaging Group
• “collect astronomical, research-quality images and make them available to the astronomical community and the general public”
Incorporation into practice
• Addresses existing problems – limited access to equipment
– attribution for images
• Reconfigurations
– Worldwide collaboration
– New modes of publishing
Pasteur’s death-bed words Bernard is right; the pathogen is nothing; the terrain is everything.
Oliver Sacks, Awakenings
Classical summative evaluation
- Quantitative only
- Little attention to antecedent conditions or classroom transactions
- Assumes fixed, knowable entities
- Cannot account for unanticipated effects
- No model for diversity of realizations
Hermeneutics
Not occasionally only, but always, the meaning of a text goes beyond its author. That is why understanding is not merely a reproductive, but always a productive attitude as well.
Hans-Georg Gadamer, Truth & Method
Alternate realizations

Implications
Technology introduction
The training model misses the most salient fact about implementation: it is a creative process involving critical analysis of the innovation’s potential… The innovation process doesn’t end, but begins with the [the user]… Since the innovation doesn’t even come into being until it is realized in an actual setting, the goal should not be to establish the endpoint… but rather, to supply the most useful tools possible for the re-creation process.
Bruce & Rubin, Electronic Quills, 1993
Evaluation
- Need to understand diverse realizations
- Innovation begins with the user
- Technology as a tool for its own re- creation
- Situated evaluation (Bruce et al., 1993; Twidale, 1993)
Design
• Design inseparable from use
• User-centered design
• Participatory design (Bjerknes et al., 1987) • Equitable relations
• An idea about technology (Menand, 2001)
Meaning of technology
only by extracting at each present time the full meaning of each present experience are we prepared for doing the same thing in the future.
John Dewey, Experience & Education
Technology studies
- Adaptive structuration: substitution, enlargement, reconfiguration (Giddens, Poole, Contractor, …)
- Longitudinal studies
- User response, reception theory
- Ecological analysis
In the traditional configuration (shi)…, tension is expressed by the curve of a roof…
François Jullien, La propension des choses