Judgment and Technology

Via Health Care Renewal, an interesting quote on the interaction between human skill and technology.  The quote comes from a paper on nursing, but I think it applies much more broadly:

Skill is the ability, drawn from education and experience, to do something expertly. It can also be defined as the effective exercise of professional judgment in non-routine situations.

Following prescribed rules, as a machine would, makes an employee competent to perform tasks, but it doesn’t make the employee skilled. They can do their job as long as there are no surprises. But when something unexpected happens, the rules break down, and caring for patients means facing the unexpected every day. Only skilled health professionals can cope with the unexpected. To know what to do, they have to rely on their own judgment. The exercise of judgment is the essence of skill.

The realm of the unexpected isn't limited to health care!

Passages I like

From Cringely:

My kids go to the best public school in Sonoma County. I know that because I chose my house based on that research. But when Cole finishes his math problems in a quarter the time it takes anyone else in the class, his teacher has him insert a wait state by putting his head down on his desk.  Conversely, when some other kid never quite gets the problem set finished, ever, well he/she never gets a rest and never masters the material, either.

The current system is unfair to both kids.

The only solution I can see is one teacher per student. And the only way something close to that is going to happen is through technology.  And it’s coming.

 A followup passage that I don't like but agree with:

My conclusion, then, is that schools serve a limited social and cultural function but our kids mainly learn despite them. My own experience is that I learned a lot about learning from half a dozen teachers in my life, so those relationships are both rare and essential. But are they reliable enough to even justify modern schools?  I don’t know. What I do know is that if I want to improve the educational environment for my children in the next year or two, I’ll probably have to come up with my own solutions.

Hardware vs. Software

The NY Times ran a story this week questioning the value of technology in the classroom.  They cite in particular a showcase district in Arizona that has been pouring money into tech without creating any observable benefit in test scores. 

To me that story highlights a key point:  The “software”, that is the teaching practices and processes that rely on tech are critical to effective utilization.  As one teacher puts it in their signature file,

“It’s not the stuff that counts — it’s what you do with it that matters.”

The conclusion to draw from that isn’t that tech doesn’t matter, it’s that you need to do more then just buy stuff.  You need to think about how to use it, and adapt as you gather experience. 

Flashback:  The Social Life of Information makes the same point about business processes.

More efforts on teacher rating

The Gates Foundation gets into the act in a big way.  I especially liked the comments on using retired principals or teachers to review video of teacher performance.  Why not expand that to parents as well?

One of the craziest mismatches in education is the number of people who want to help their kids vs. the number of people who can improve what goes on in school.  Video allows time-shifting and space-shifting, vastly expanding the pool of people a principal could draw on.

SharePoint Adoption

Economics of Information noted a recent survey showing business users by and large ignoring Microsoft’s structured file sharing service, SharePoint.  They ask why people still use email to collaborate, despite its nominal inefficiencies.  Here are my thoughts based on my experiences in the finance industry:

• Disclosure matters.  Having a simple, understandable-by-a-judge document that says A sent X to B on such and such date is critical on a number of levels.  It can prove whether a contract was respected or violated or whether or not a department or employee did it’s job correctly or incorrectly.  And yes, intra-company conflicts are no less bloody or consequential even if lawyers are not involved.  SharePoint nominally accomplishes this, but with more ambiguity.  Say a file changes, how do people know what changed and whether it’s pertinent to them?  How do you set policy so that you even know whether everyone knows that a file changed?  Unless you are very, very careful in how you set policy you lose knowledge about disclosure.  Most managers don’t have time to figure that out and won’t take the risk in adopting something could reduce their control of process.
• Collaboration in the creation of office documents is over-rated.  Most technical documents have a two step life.  An analyst level person creates the document and a manager reviews and makes changes as necessary, and it’s done.  When I have seen manuscript type documents where you need multiple managers reviewing/editing a single document, it’s almost always a formal sign-off process where you need documentation that everyone reviewed and approved.  I could imagine SharePoint doing that but again you’d need a lot of policy work to automate the documentation, this time cutting across departments.
• Another note on collaboration- I’ve had SOX auditors tell me they don’t like seeing office documents involved in material financial calculations at all.  I can’t imagine them being comfortable with something that introduces even a shred of additional ambiguity.  SOX compliance is another reason managers are probably reluctant to change process.

All of this is to say that you need a lot of policy work to control notification and documentation of such.  That can be done but requires an investment of time by managers, and the payoff compared to using email doesn’t cut it.

If I were running a startup I’d demand people use a structured file system with lots of opportunity for markup information (what is this, what does it pertain to etc.) and I’d expect businesses that use it to have a considerable advantage down the road (how much time do you spend trying to untangle the work of predecessors?)  I would not expect established businesses, especially big ones with lots of history, to lead the charge.

Wireless Responder Networks

Why do politicians crave wireless networks for responders?  Is there an example anywhere, anywhere in the country where one of these projects produced any kind of positive return on the money dumped on them?  Thanks to the passage of ballot measure 26-117 I get to pay for TWO of these fiascos.  Joy.

School Performance Data: an untapped resource

Patrick Emerson writes about school reform.  These lines stood out to me:

What is needed is a little humility and a desire to learn what works on the part of policy makers.  PPS, for example, is sitting on a mountain of incredibly valuable data, but as far as I am aware no real study of this data is being done.

Data indeed!  Every homework assignment, quiz, or test creates comparative data by student and class.  The volume of that data compared with standardized tests is overwhelming.  What happens to it?  Aside from being added to some semester average that ultimately results in a letter grade, what use is made of it?

Success stories like Bridger or Pauling show good things can happen when you actually use that data by making coursework contingent on performance, constantly evaluating and adjusting as you go.  Those schools make coursework dynamic, teaching students what they need to learn versus what the teacher thought the class needed to learn when they set lesson plans at the beginning of the year.  That is what information technology does best- making something specific and relevant instead of general and irrelevant.

Information Technology at it’s best…

Information Technology at it’s can best makes things which more specific and relevant, instead of general and irrelevant.  Think about what you watch on Hulu or On-Demand Cable, vs. the old days of network TV.  Here we see IT applied to non-profit organizations.  By making donors more informed their donations can more effectively serve their interests, and that is a good thing.

IT in Hospitals

Interesting review of the impact of IT investment by hospitals.  Bottom line- without business process improvement better tech doesn't help.  That jives with work by Brynjolfsson & Saunders that real value lies in improving business process and tech in tandem, whereas the former in practice tends to lag the latter by many years.

My own two cents:  Process improvement can't be done by IT folk.  You can't properly re-engineer someone's business if you don't know it, and by the time IT folks learn it the tech will be outdated.  Better to put a system up fast, and then give the people who actually do the work pathways and incentives to improve their own processes.  Let it evolve, with supervision.  It's easier to teach them a little tech then teach a tech person a lot of [whatever].

Another chapter from the book of IT Hell

It isn't funny seeing a story like this in times like these.  Their finance and IT group employs all of 22 people.  How could that have ever made sense?

POW: Clay Shirkey

I just got around to listening to a very good lecture by Clay Shirky sponsored by LSE.  Shirky discusses his concept of "Cognitive Surplus".  The gist of it is that by converting a small sliver of the time otherwise spent watching TV (passively consuming) into active creation we are able to reap enormous benefits.  For example, the total number of all hours spent creating wikipedia is less then the time we spend watching commercials on any given weekend. 

The internet thus far has provided mostly private benefits.  Those interested in a particular topic are empowered to discover and create content virtually cost free, excluding time.  That isn't a bad thing, but what Shirky looks forward to is projects with a global impact- voluntary collaborative efforts that fundamentally change cultures.

This was an entertaining and interesting lecture, I strongly recommend it.

Fans of Shirky's work can find an Econtalk interview from 2008 here.

Another good Cringely piece

Cringely has a followup to his column on consultants, this time tackling the big research firms like Gartner and Forrester.  His critique in a nutshell:  Those firms live off churn, and that is not in anyone else's interest.  Worth reading.

IT Consultants strike again!

This time the victim is the City of Portland.  $20 million goes a long way here.  I'd surely like to know who to blame for this, Potter doesn't cut it since he's leaving anyway.  Who decided on Ariston?

IT Gone Bad

Related to Cringely's comments on IT, here is a story about a large non-profit in the Portland area at risk of bankruptcy ostensibly because of bad IT.  That's disgusting!

POW: Cringely

Robert Cringely put out an excellent column on 3/21/08, the podcast is here.  The theme:  Society at large is only just beginning to adapt to the emergence of personal computing, and that changes to come will be greater then the ones we've already seen.  He picks on schools in particular as an institution destined to be radically altered in character and purpose.

The current educational system is designed to help children internalize knowledge, that is to learn.  Technology in this setting is a two edged sword- on the one hand it permits vast and cheap access to knowledge (Wikipedia).  It also permits vast and cheap plagiarism and cheating (Yahoo Answers).

Cringely poses the question:  Should schools be fighting such conduct?  Is knowing how to find an answer an adequate substitute for knowing the answer itself?  In an information economy the answer must be yes, and if so what then should educational institutions look like?

For a less pessimistic view of the future of education, see Brown and Duguid.

Japanese not adopting Vista as hoped

Apparently the Japanese are less then enthusiastic about Windows Vista...

POW: TWIT #134, how to build a better podcast

My podcast of the week is episode #134 of TWIT (This Week in Tech).  One of the guests was David Winer, one of the key architects behind RSS.  About 20 minutes in the guys get into a pretty good discussion of podcasts, and the shortfalls of their current use and user experience.

David notes that podcasts are heard primarily on personal MP3 players, which are designed around songs.  Songs differ from podcasts in a number of ways, principally that you typically only want to hear a podcast once and then get rid of it.  Songs are typically kept and reused, listening to them doesn't subtract from their value.  It got me thinking about ways the experience of hearing a podcast could be improved.

First, long podcasts need breaks.  TWIT has been running almost 90 minutes lately, I never listen to the whole thing at once.  Bookmarks allow me to stop and later re-start, but in doing so I lose something, I lose the context of the conversation.  It would be helpful as a listener to have 'producer' bookmarks, like a table of contents, to know where one section stops and another starts.  You could argue the solution is that I just pay more attention to the podcast.  After all, I can stop reading a book and generally pick it up later and still know what is going on, but that comparison is misleading.  The reality is that I almost always use podcasts as an audio background while I do something else (like I use radio).  That's in contrast to reading a book, in which reading is the primary activity and I actually try to minimize what is happening in the background.  By paying more attention while I read I absorb more, with podcasts I might pick up a few interesting facts but it's primary function is entertainment.  So to summarize, long podcasts are broken up by the listeners already, it would be helpful if the podcast producers took this into account and included their own bookmarks (and recorded with them in mind) to create more listener friendly breaks.

Second, we need a podcast player that recognizes the producer bookmarks and is able to automatically jump to new material.  I'm thinking of an application that can mimic radio- integrating podcast material, regular MP3 songs, and cut music/sounds to transition between the two.

Finally, there needs to be a better way to get advertising integrated.  Why not add commercial spots that could be fit into the producer bookmarks?  Current practice is to have podcasters plug their advertisers.  On TWIT for instance, they always talk up audible.com, typically mentioning a specific title they recommend.  Why not allow outside producers to make a spot to sub in?  I'd prefer to hear directly from the author of a book (or their representative) talk about why I should download their work, rather then hear Leo Laporte talk about it.  Podcasters could have parallel feeds- one with ads, one without, and let listeners choose which to subscribe to.  That way no disaffected fans.  I think there may even be a cottage industry in there- the podcast-advertisement, wherein content creators like authors or musicians have a minute or two to tell listeners why they should purchase the product. 

Why would I care about advertising?  Because I think podcasters should be able to make a living (and make more podcasts).  I'm thinking about Bob Packett selling T-Shirts or Reilly and Markham and coffee mugs.  How much more sense would it make for them to run spots with history books?

Jumbo laptop

I don't think this would fit on my desk, but it is still pretty cool.

Book Analysis and Notes: Social Life of Information

I've added some notes on the Social Life of Information, by Brown and Duguid. I still don't really remember why my Dad recommended it. The book had a lot of goodness, showing how social context and interactions affect the absorption of information, and that the internet can't be all things to all people because of this. There is also a certain amount of jargon (deconstructing the 6 D's) that I didn't care about.

Mostly I felt a lot of nostalgia for the 90's while reading this- a time of near infinite possibility and no fear.