Technology Blog

OK, I put up with the numerous bugs, the frequent nondescript error messages, and so on, but tonight I tried to just click on the Groups link in MySpace and got this, repeatedly for the past half-hour or so:

Hey Rupert… how about throwing a couple of extra million into fixing the infrastructure!!!

This post was written by Scott Allen, source: The Real Problem with MySpace

Martha Brogan, Contexts and Contributions: Building the Distributed Library, Digital Library Federation, November 22, 2006.  A major study (282 pp) of OAI interoperability from the DLF, one of the organizations, with CNI and NSF, that originally sponsored the development of the OAI-PMH.  Excerpt:

This report updates and expands on “A Survey of Digital Library Aggregation Services,” originally commissioned by the DLF as an internal report in summer 2003, and released to the public later that year. It highlights major developments affecting the ecosystem of scholarly communications and digital libraries since the last survey and provides an analysis of “OAI implementation demographics,” based on a comparative review of repository registries and cross-archive search services. Secondly, it reviews the state-of-practice for a cohort of digital library aggregation services, grouping them in the context of the “problem space” to which they most closely adhere. Based in part on online survey responses collected in fall 2005 from an online survey distributed to the original core services, the report investigates the purpose, function and challenges of next-generation aggregation services. On a case-by-case basis, the advances in each service are of interest in isolation from each other, but the report also attempts to situate these services in a larger context and to understand how they fit into a multi-dimensional and interdependent ecosystem supporting the worldwide community of scholars. Finally, the report summarizes the contributions of these services thus far and identifies obstacles requiring further attention to realize the goal of an open, distributed digital library system.

The new report aims to inform DLF’s continuing efforts “to foster better teaching and scholarship through easier, more relevant discovery of digital resources, and a much greater ability for libraries to build more responsive local services on top of a distributed metadata platform,” as articulated in its successful IMLS National Leadership Grant, “The Distributed Library: OAI for Digital Library Aggregation.” Extending over a two-year period from October 2004 through September 2006, the grant enables DLF to prototype a “second generation” OAI finding system. Concurrently, it affords DLF the opportunity to address challenges identified in the 2003 survey and voiced by early OAI adopters. In particular, DLF is building a comprehensive OAI registry, establishing best practices for shareable metadata, improving communication between data and service providers, and developing curricular materials and training sessions to introduce OAI best practices to a widening circle of institutions (Shreeves et al. 2005).

Using the 2003 survey as a point of departure, this companion report takes a fresh look at the evolution of interoperability and federating heterogeneous content, especially as realized through implementation of the OAI protocol. It re-examines the original set of digital library aggregation services as well as representative new initiatives in an effort to identify trends - progress, needs, and challenges. How are they evolving over time? What have they achieved? What is impeding their progress? How do they envision their future? An online survey conducted in fall 2005 gathered baseline information from more than forty aggregators….

See especially these sections:

• 2.2, Open Access Ascendant - Growth of OAI-compliant Repositories
• 4.1, Points of Reference: Open Access and the Open Archives Initiative
• 4.2, Links in the Scholarly Communication Value Chain

source: Major report on OAI interoperability

Steven William Glover and four co-authors, A review of health and access to health information in Bhutan, Health Information & Libraries Journal, November 29, 2006.  Not even an abstract is free online, at least so far.

source: Access to health information in Bhutan

Oxford Journals offers faster and better access to metadata records with OAI-PMH functionality, a press release from Oxford Journals.  Excerpt:

Oxford Journals, a division of Oxford University Press, today announced that all abstracts and metadata for over 180 journals can now be obtained using OAI-PMH functionality.

The Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH) offers any third party, including aggregators and libraries, the opportunity to acquire metadata records within a standard format.

The Oxford Journals OAI-PMH repository contains abstracts and metadata for all journal content, including the Oxford Journals digital archive, with records dating back to 1849. The repository is constantly updated with current issues, as well as articles published as Advanced Access, and provides a convenient location for harvesting metadata collections, or single article metadata.

“Providing OAI-PMH functionality offers third party aggregators and librarians a vastly improved way to extract metadata records from our content”, commented Pam Sutherland, Journals IT Director, Oxford Journals. She continued,

“Metadata can be harvested at any time, as frequently as required, and it’s now much quicker to access the data than via other mechanisms. The Oxford Journals repository is extremely flexible, and users can set their own parameters to obtain precise sets of metadata, sorting by journal, date, volume, or issue. A date-stamp has also been included within all records, even though this is an optional element of OAI-PMH, to allow harvesters to select all records from a specified date.”…

Comment.  This is smart and all journals (OA and TA) should do it.  (I’ve been recommending it since 2004.)  Inderscience seems to have been the first non-OA publisher to test its potential as a more effective and less expensive alternative to traditional marketing.  For details, see this Inderscience case study from 2003.

source: Oxford supports OAI-PMH harvesting of its journal metadata

Nick Farrell, Internet Archive is free from DMCA, The Inquirer, November 30, 2006.  Excerpt:

After a long and expensive fight, the Internet Archive has been granted an exemption from the Digital Millennium Copyright Act (DMCA).

The Archive stores computer games and other old software for posterity, and runs the famous Internet Wayback machine. However, it feared that because it was copying others’ work it could be in breach of the DMCA.

According to [a press release at] the outfit’s website, thanks to the hard work of two law school students of Peter Jaszi of American University, Jieun Kim and Doug Agopsowicz, the Internet Archive and other libraries may continue to preserve software and video game titles without fear of going to jail.

The move cost it $50,000 of pro-bono lawyer time, and a spokesArchivist said that it would be jolly nice if the government could sit down and write a better law that could protect libraries and others archivists of the World Wide Wibble.

source: Limited copyright exemption for the Internet Archive

Rufus Pollock, UK National Statistics: Are They Open or Not? Open Knowledge Foundation Weblog, November 30, 2006.  Excerpt:

I’ve used data a couple of times from the UK’s national statistics site.

The other day I went there to investigate their licensing as part of an effort to do a simple survey of the openness of various UK government agency’s data. To summarize their copyright statement (full details are in 1):

• National statistics are under Crown Copyright
• If you want to reproduce the statistics other than for research or private study you need a ‘core’ (now called PSI) click-use license

This is where it gets interesting. According to the PSI license (which I’ve textified and posted [here]):

“In this Licence, to reproduce includes the following non-exclusive rights throughout the world:

6.1 publishing the Material in any medium. This includes featuring the Material on websites which can be accessed via the internet or via an internal electronic network or on an Intranet;”

This seems to be pretty broad and cover anything I’d like to do….If this licence was effective it would mean that the data was open.  However further down the National Statistics copyright page one finds:

“Customers wishing to repackage and/or redistribute National Statistics material in their own products or services and allow their customers to use such material should contact msolicensing@cabinet-office.x.gsi.gov.uk with details of their request.”

This seems to imply that repackaging and redistribution require separate permission and are not covered by the PSI licence. If this is the case the freedom to reproduce granted in the click-use license is rendered completely void. All of this makes me wonder whether the National Statistics UK are able to see some special distinction between ‘reproduction’ and ‘repackaging/redistributing’ that’s not apparent to the rest of us.

Comment.  One of the chief benefits of making information OA is that users are spared the expense and delay of asking permission to make use of it.  Some providers understand this and provide blanket permission for certain uses in advance.  However, what they don’t always realize is that this benefit is completely negated when the permission is vague or inconsistent.  Then conscientious users will still have to ask permission, denting their productivity, sometimes denting their budget, and increasing the unrelenting pressure to become less conscientious.

One of my favorite examples is a June 2004 policy by Johns Hopkins University Press (JHUP) allowing authors to self-archive JHUP journal articles in their institutional repository “provided the [repository] does not directly compete with either the Johns Hopkins University Press or Project Muse.”

source: Permission granted and withheld

The Free Our Data blog has two posts today on how Denmark and Australia charge for access to publicly-funded government data.

source: Access models to govt data in Denmark and Australia

Although John Willinsky’s book, The Access Principle (MIT Press 2005), already had an OA edition, Willinsky has deposited another OA copy in dLIST.  The updated abstract:

This work is copyrighted by MIT Press and licensed under a Creative Commons Attribution-NonCommercial 2.5 License. MIT Press has granted permission to place a copy in dLIST. Readers can also purchase the book from MIT Press, which publishes it (see alternative location for details).

Following abstract is from MIT Press: Questions about access to scholarship go back farther than recent debates over subscription prices, rights, and electronic archives suggest. The great libraries of the past — from the fabled collection at Alexandria to the early public libraries of nineteenth-century America — stood as arguments for increasing access. In The Access Principle, John Willinsky describes the latest chapter in this ongoing story — online open access publishing by scholarly journals — and makes a case for open access as a public good. A commitment to scholarly work, writes Willinsky, carries with it a responsibility to circulate that work as widely as possible: this is the access principle. In the digital age, that responsibility includes exploring new publishing technologies and economic models to improve access to scholarly work. Wide circulation adds value to published work; it is a significant aspect of its claim to be knowledge. The right to know and the right to be known are inextricably mixed. Open access, argues Willinsky, can benefit both a researcher-author working at the best-equipped lab at a leading research university and a teacher struggling to find resources in an impoverished high school. Willinsky describes different types of access — the New England Journal of Medicine, for example, grants open access to issues six months after initial publication, and First Monday forgoes a print edition and makes its contents immediately accessible at no cost. He discusses the contradictions of copyright law, the reading of research, and the economic viability of open access. He also considers broader themes of public access to knowledge, human rights issues, lessons from publishing history, and “epistemological vanities.” The debate over open access, writes Willinsky, raises crucial questions about the place of scholarly work in a larger world — and about the future of knowledge.

John Willinsky is Pacific Press Professor of Literacy and Technology at the University of British Columbia. He is the author of Empire of Words: The Reign of the OED and a developer of Open Journals Systems software.

source: Willinsky book now self-archived

Online Educa tackles lifelong learning and open access, a press release from JISC, November 30, 2006.  Excerpt:

Delegates have been gathering for the world’s largest e-learning conference which began in Berlin yesterday. The Online Educa conference on Technology and Supported Learning, now in its twelfth year, has attracted nearly 2,000 delegates from education, research, Government and commercial sectors from around the world….

Joint activities by JISC and SURF began in the afternoon with a workshop on open access. JISC Scholarly Communications consultant Fred Friend introduced the topic, saying that there were considerable economic, social and educational benefits to making research and other outputs available without financial, legal and technical barriers to access. It was also, he suggested, a question of giving more power to authors.

He outlined the two complementary routes to open access. Self-archiving or the deposit of a pre-print or post-print in an institutional or subject repository is one route, he said, while the other is publication through an open access journal….Both routes, suggested Fred Friend, will enable the opportunities provided by technological developments, such as data mining and text mining, to be realised….There are significant economic benefits to open access too, he continued, through the sharing of developments in medical research, for example, and through the value for money gains of publicly funded research being more freely available….

Amber Thomas, JISC programme manager spoke about JISC’s work to encourage the establishment and development of repositories in the UK….

Erik Saaman of SURF spoke about open access initiatives in the Netherlands, such as DAREnet (research), LOREnet (higher education), Edurep (non-HE) and Driver (European), and the common technical architectures which underpin them….

Ryan Hargreaves gave an overview of Jorum, the free online repository for learning and teaching resources which is aiming to create a culture of sharing of learning and teaching resources….

John May of the SURF Foundation spoke about LOREnet which is aiming to create a similar ‘community of sharing’ in the use of online teaching materials in the Netherlands….

source: The OA workshop at Online Educa Berlin

Paul Kobulnicky, Scholarly Reputations: Who’s Got Buzz?  Educause Review, November/December 2006.  Excerpt:

While giving a presentation at a recent international meeting on institutional repositories, I responded to a member of the audience who questioned the motivation for faculty to participate in such repositories. The individual repeated the oft-mentioned position that scholars, by placing publications in such repositories, unduly risk their contractual relationships with publishers. I responded by acknowledging the concern and by describing options that minimize risk, but since I had the stage, I also wondered aloud about which “big win” choice scholars would make as time moves forward. Those of us who believe both in the rigor of peer review and in open access know that these are not mutually exclusive choices but rather are mutually beneficial elements in advancing scholarship. However, for the sake of the argument, I asked if scholars would choose (1) to have their work published in the premier journal in their field or (2) to have that work regularly come up on the first screen in an appropriate Google search. Although I made the statement to stimulate discussion in front of an audience, I believe that my straw-man choice raises issues that need to be discussed further….

[T]he choice between a prestigious scholarly publication and a high Google ranking was not really an either/or choice. No one should dismiss or diminish the value of using high editorial standards to point both the scholarly and the general communities to work of high quality. However, the placement of scholarship in an open-access institutional repository need not preclude publication in a prestigious title. Authors and their publishers can move to provide open access to their works in pre- and/or post-publication versions. From the perspective of return on social investment, open access to published works can create positive returns for both authors and scholarly publishers. My straw-man choice was contrived, of course: neither scholarship without perceived relevance nor scholarship evaluated only on Web buzz is desirable.

Within the classic and respected framework of scholarly review and selection, scholars need to think about the impact of contemporary, open information access and about the mechanisms that can be used to generate relevance in their work. They need to think about choosing appropriate metadata and about how others, not necessarily just their disciplinary peers, might seek and use their information. They need to think about the links that ought to exist between their work and the broader society that inevitably supports it. Finally, they need to give some thought to the very unscholarly but pragmatic aspects of their work: marketing and promotion, which contribute to the creation of scholarly reputations with “buzz” on the Web.

source: Traditional scholarly prestige and new forms of buzz

source: ERCIM signs the Berlin Declaration

Dick Kaser, Funding Open Access, Information Today blog, November 29, 2006.  Excerpt:

I had a fascinating conversation at lunch with Anthony Watkinson (center), a publishing consultant and part-time lecturer at University College London. Just back from The Charleston Conference where he spoke on “The Future of Publishing in an Age of Uncertainty,” he engaged me in a discussion about Open Access publishing.

I was not taking notes, but the bottom line seemed to be that Open Access is a publishing model still in search of a business model . . . or at least a sustainable funding base.

Though many institutions have pledged support of the idea, Watkinson observed, few have come forward with the money to pay for publishing papers in such a way that they can be offered free-of-charge to users.

Comments.  Depending on how Anthony fleshed out this comment, I could agree or disagree.  More specifically:

• Today the Directory of Open Access Journals lists 2,480 peer-reviewed open-access journals.  That’s about 10% of the whole in about 10 years.  Print and toll-access journals have had about 350 years to reach their current level of penetration.
• A large minority of these OA journals (about 47%) are funded by author-side fees, usually paid by the author’s research grant.  The majority are no-fee journals, supported by institutional subsidies and other methods.  These journals have business models.  Are they adequately funded?  Some probably are and some probably aren’t, just like subscription journals.
• Long-term, the best source of funding to pay for OA journals is the money now spent on TA journals.  So we’re not facing an economic mystery, just a complicated choice.
• To many informed observers, the subscription model is doomed.  For example, a public letter (January 2004) signed by the head librarians of the 11 campuses of the University of California, and Lawrence Pitts, Chair of the UC Academic Senate, asserted that the subscription model was “incontrovertibly unsustainable”.  The uncertainties of OA business models must be considered together with, or compared to, the uncertainties of TA business models.

source: Funding peer-reviewed journals

I had a bit of an epiphany while chatting to Antoin
about the qpsmtpd/EC2 idea. Craig
had the same thoughts.

Here’s the thing — there’s actually no need to offload the SMTP part at all.
That stuff is tricky, since you’ve got to build in a lot of fault tolerance,
quality-of-service, uptime, etc. to ensure that the MX really is reachable.
Since an EC2 instance will lose its “disks” once rebooted/shut down, you need
to store your queues in Amazon S3 — which has differing filesystem semantics
from good old POSIX — so things get quite a bit hairier. On top of that, it
requires a little RFC-breakage; there are issues with using CNAMEs in MX
records, reportedly.

However, if we offload just the spamd part, it becomes a whole lot simpler. The
SPAMD
protocol
will work fine across long distances, securely, with SSL encryption active,
and SpamAssassin will work fine as a filtering system in an entirely stateless
mode, with no persistent-across-reboots storage. (What about the
persistent-storage aspects of spamd operation? There’s just the
auto-whitelist, which can be easily ignored, and I haven’t trained a Bayes
database in 2 years, so I doubt I’ll need that either

If the spamd server is down or uncontactable, spamc will handle this and retry
with another server, or eventually give up and pass the message through, safely
intact (though unscanned).

Given that there’s a cool third-party ClamAV
plugin now available for
SpamAssassin, this system can offload the virus-scanning work, too.

So here’s the new plan: run the MTA, MX, and the super-lean “spamc” client on
the normal MX machine — and offload the “spamd” work to one or more EC2
machines.

Basically, there would be a CNAME record in DNS, listing the dynamic
DNS names of the EC2 spamd instances. Then, spamc is set to point at that
CNAME as the spamd host to use. As EC2 instances are started/removed,
they are added/removed from that CNAME list and spamc will automatically
keep up.

Pricing is reasonably affordable — don’t send over-large messages to the EC2
spamd; rate-limit total incoming SMTP traffic in the MTA; and use the SPAMD
protocol’s REPORT verb to reduce the bandwidth
consumption of mails in transit by ensuring that the mail messages are only
transmitted one-way, MX-to-EC2, instead of both MX-to-EC2 and EC2-to-MX.
That will keep the bandwidth pricing down.

Recent figures indicate that I got about 90MB of mail per day, at peak, over
the past weekend (which nearly DOS’d my server and caused some firefighting) –
68MB of spam, and 13MB of blowback. At 20 cents per GB, that’s 1.8 cents per
day for traffic. Plus the $0.10 per instance hour, that’s $2.42 per day to run
a single EC2 instance to handle DDOS spikes. Of course, that can be shut down
what load is low.

Yep, this is looking very promising. Now when are Amazon going to let me
onto the beta program for EC2?…

Tags:amazon aws ec2 spamassassin spamc spamd

This post was written by Justin, source: SpamAssassin as an EC2 service

Canada’s Athabasca University has adopted an OA policy.  (Thanks to Virtual Canuck via OA Librarian.)  The policy:

Athabasca University requests that academic and professional staff deposit an electronic copy of any published research articles (as elsewhere accepted for publication) in an Athabasca University repository. The contract with the publisher determines whether the article is restricted (lives in the repository as a record of the University’s research but is not accessible online by searchers) or open access (accessible online by searchers).

From Terry Anderson at Virtual Canuck:

We argued that this policy was consistent with Athabasca’s commitments to access as “Canada’s Open University”. We had originally hoped that the verb “request” could be strengthened to “requires” but the fears of curtailment of so-called “academic freedom” scared some away and we compromised on the weaker “requests”. To support self-archiving, the AU library has installed a version of MIT’s DSpace known locally as AUSpace…

I was somewhat surprised at the lack of understanding of Open Access publishing and of the need for action on the part of individual scholars to insure that works funded by public funds are made available to the public. The hundreds of thousands of dollars spent annually by even small Universities like Athabasca on database services that provide full text of many (most) proprietary publications has lulled many academics into a sense that such services are available to all scholars. This is obviously not the case and especially not in areas of the 3rd world….

From Heather Morrison at OA Librarian:

Congratulations to Athabasca University for taking one small step forward. However, this is a weak policy, for two reasons, and I would strongly suggest that other universities considering an open access policy not emulate this one.

First, it requests rather than requires academic and professional staff to deposit published research. Experience has shown that policies that request deposit simply do not work, whereas policies that require deposit are extremely effective. Also, rather than beginning with voluntary compliance, the nature of open access is that it makes more sense to start with the requirement - because as soon as faculty and staff experience the impact advantage, and other advantages of convenience, with open access, from a requirement, they will quickly come to appreciate open access, and so they will happily voluntary self-archive.

Second, it leaves the question of open access up to the publishers. Even setting aside the rights of the university, its students and alumni and taxpayer funders at a public institution like Athabasca - universities wishing to educate faculty members about open access would do well to also advise faculty members about their rights, too. Authors and scholars do not need to give up their copyright in order to have their work published; the only copyright that an author need give a publisher, is simply the right to publish; authors can do this and retain their copyright, too.

The vast majority of traditional publishers already routinely allow self-archiving of preprints or peer-reviewed postprints, or both….

As a proud former student [of AU], I would like to point out that, if Athabasca University’s new open access policy is not exemplary in the area of self-archiving, AU has long been a leader in open access publishing, as the home of ICAAP, the International Coalition for the Advancement of Academic Publishing….

source: Athabasca University asks faculty to self-archive

source: Nokia World - Creators Panel

source: Nokia World - Vanjoki keynote

Chris Greer, The Digital Data Universe of the Future, a webcast of a 56 minute lecture at the Library of Congress, October 11, 2006.  (Thanks to ResourceShelf.)  From the blurb:

Dr. Chris Greer of the National Science Foundation discusses efforts of NSF to develop a strategic vision that provides a national digital framework in which NSF can work with partners in public and private sectors to address data acquisition, access, usage, stewardship and management challenges in a comprehensive way. NSF’s five-year goal is twofold: 1) To catalyze the development of a system of science and engineering data collections that is open, extensible and evolvable; and 2) To support development of a new generation of tools and services facilitating data mining, integration, analysis, and visualization essential to turning data into new knowledge and understanding.

Speaker Biography: Dr. Chris Greer’s current responsibilities include strategic planning for cyber infrastructure for the biological sciences and digital data activities in the newly formed Office of Infrastructure of the National Science Foundation. He received his Ph.D. from UC Berkeley and did post-doctoral research at the California Institute of Technology.

source: Open data at the NSF

What a strange day. Nearly 70 degrees at the end of November.

Our front door is wide open, as are the bedroom windows. Across the street, our neighbors are washing and vacuuming their cars. As is their wont, they’ve got a radio blasting music loud enough that we can hear it in the kitchen—and today the radio is playing Christmas songs. It’s almost surreal to hear Blue Christmas blaring through open windows on a balmy afternoon.

source: not-so-wintery weather

The University of Alicante in Spain has signed the Berlin Declaration on Open Acces to Knowledge.

source: Spanish University signs Berlin Declaration

Open Access Anthropology(but is now dormant) and the second in LIS.

source: Another field-specific blog on OA