Ask not what your open source project can do for you...

I sympathize with Sharon's plight about Drupal documentation and Clancy's concerns. As with many open source projects, the documentation could be much improved. That's a given that most open source projects recognize.

Unfortunately, I've seen similar observations about the need for better documentation posted to the Drupal forums many, many times. The solution to the problem is clear--many long term Drupal contributors respond to those that note the documentation problems with the best solution--but those who could provide the solution almost never do.

Many of you reading this who are open source software users probably think I'm talking about the software developers, that they should be the ones to improve the documentation. I'll admit it is a help when they are involved, and sometimes their assistance is needed for clarifying how the software works, but they aren't the ones that should be taking action. To paraphrase JFK, ask not what your open source project can do for you, ask what you can do for your open source project.

Most larger open source projects are a lot like not-for-profit style organizations, and in fact, often form non-profit foundations to handle monetary donations. Like Wikipedia and every other commons based peer production project, development of the product depends on the users. One has to escape beyond thinking of open source software as just a product and see open source as about communities where volunteers provide coding, bug testing, documentation, and support. All too often, there are volunteers to do the coding and the bug testing, but not nearly enough to offer support, and documentation comes dead last in the area of willing contributors. This despite the fact that creating and maintaining documentation is a massive task, much more difficult than people who have not been involved can imagine.

Open source software is often free, and free is good here because we have the opportunity in open source to shape the product in a way that we cannot with the proprietary software imposed upon us by our institutions. Writing teachers who use open source software may not be good coders, but they can be excellent contributors of documentation. In my years of working with Drupal and with other writing teachers who use Drupal, I haven't seen many contributions from other writing teachers to drupal.org's documentation. My advice is don't just expect that the soup kitchen with free meals should provide you with better food. Go find better food donations for the soup kitchen or get in there and help prepare the meals so that they taste better.

Consider this a challenge. Give some of your time to an open source project in the same way that you do other types of community service. If you find the documentation wanting for what you are trying to accomplish, take notes as you figure it out. Then afterwards, revise the existing documentation or provide new text as needed and negotiate with the project to have your changes included. Your help will be much appreciated, and it's an opportunity to learn to collaborate in the creation of text on commons based peer production projects. At the same time, you'll be helping to build the public commons and make things easier for those that come behind you.

Comments

The problem is not that the help documents do not exist, mostly.

The problem is
(a) that the Drupal website is very badly organized, and
(b) searches for Drupal help are generally futile, and
(c) help that is located is often obsolete or wrong

In other words, the solution is out of our hands. We do not control, and cannot change, the Drupal website. And therefore we cannot fix the documentation problem.

I documented many of the specific instances in my 13-part series on installing Drupal last year (that it took 13 parts is an indication of the problems I was facing - yes, I was trying to do a lot of non-standard stuff - but the point is that it gave me a very clear view of the problem).

The problem is, to be more specific:

(b) searches for help on the Drupal site typically result in lists of meaningless and useless discussion pages, rather than anything like documentation, and

(a) even when you land on a page that might be useful, such as a module distribution page, help is not available from that page

(c) and when you actually find some help, it frequently refers to a module that is now obsolete or process that no longer exists

I don't know why, but search engines are utterly unable to distinguish between what is important on the Drupal website and what is utterly trivial. Hence, a search for something specific displays a long list of meaningless chatter, rather than a help page.

They are not helped at all by the Drupal website. Consider, for example, the help page at http://drupal.org/node/363 (this was chosen at random; it is completely typical). Note first that the URL offers us no help. The page title is "Basic site configuration | drupal.org" (everything is 'drupal.org').

On the page, terminology is inconsistent and vague. For example, "You will want to start your configuration with your drupal site's basic settings can be found on the settings page, which you can reach by clicking administer » settings in the navigation block." Clicking the link does not take you to 'basic settings' but rather 'Settings'. This page contains another link to 'General Settings' (note that you still haven't found anything) and very advanced topics, such as 'clean URLs' (where you are helpfully told "this works only for Apache servers which have the LoadModule rewrite_module configured and mod_rewrite enabled in httpd.conf configuration file" (no links).

On the 'General Settings' page ( http://drupal.org/node/43794 ) we are told how to set the site name, email address, slogan, etc. But nowhere on this page are we told that these options are reached by clicking on by clicking administer » settings. If we were searching for a way to change the slogan, even if we landed on this page, we would be completely stymied (in fact, of course, the search result for 'slogan' http://drupal.org/search/node/slogan does not take us anywhere near this page).

Now it's all very easy to say that people should fix this. But there is a lot more to simply writing some decent documentation needed to solve these problems. Specifically:

(a) The Drupal website needs to be split from the Drupal developer community website and the Drupal user community website. It is just a bad idea to mix up discussion threads with documentation, especially when the volume of discussion dwarfs that of the documentation.

(b) Drupal elements (such as, say, the 'Administration Menu') should be named, and this name used in both titles of documentation pages and links to those pages.

(c) Help pages on the Drupal.org should be rewritten to provide help on the specific topic. Boilerplate links to generic pages (something very common in the module documentation) should be avoided. Links that confuse (such as 'There are many other places to configure Drupal.') should be avoided. Self-congratulation ("A lot of the information you are asked to supply on the settings page is self-explanatory") should be eliminated.

(d) Drupal's internal search should be altered to give priority to certain types of documents (such as help pages) - despite the software design, not all nodes are equal.

(e) Documentation for different versions of Drupal should be on different websites or should be very clearly marked as such (colour-coding documentation for each version would be a good idea).

(f) Modules should be required to pass compatability tests before being included in Drupal's documentation or search results. Documentation and code for modules that have been abandoned or obsoleted should be removed from the site.

(g) Other solutions, which I may not have thought of.

Now - importantly - a typical open source user can't just walk in and make changes like this.

Even if they created perfect documentation, it would not appear on Drupal's website unless the website was radically redesigned, which is not likely to be done at the request of a typical user.

And even then, splitting the site apart, changing the search code, etc., are well beyond any sort of fixes the average user can provide.

The fact is, the people at the top (or the core, or the heart, or whatever) of the Drupal project will have to address this. Nobody else has the influence or expertise to do so, and very few people have the years of dedicated service it would take to attain such experience and influence. That's just a fact.

The problems with Drupal documentation are systemic, and because very little attention has been paid to the issues that have been widely cited, the problems persist. people - even people who devote the bulk of their time to open source - are not willing to pursue solutions in this light.

The problem is that there are not people to do the sort of massive rewriting and reorganization of the documentation you describe on drupal.org. I suspect that this is a problem with many open source projects having myself observed first hand how and why it is a problem at Drupal.

As for many of your suggestions for Drupal, I haven't been involved with the documentation team lately, but some of them have been discussed in great detail. A few improvements have already been made. For instance,

  • When documentation exists for individual modules in the handbook, there is a link from the module information page (e.g., see the read documentation for the event module).
  • The search engine is better at finding relevant results, and the advanced search option now provides many options to filter results, such as search by node type.

And I must disagree. There are plenty of ways that individuals can get involved revising existing documentation even if they can't address bigger concerns such as site reorganization.

So I'll return to one of the the main points of my post. I'm sure that most open source projects which could use improved documentation know it, and they probably know many of the shortcomings. I saw plenty of good ideas for improving documentation at drupal.org on the discussion list. And there was tons of discussion from new people in the forums about how it could be better. But unfortunately, there are many, many more people critiquing what's wrong with the documentation or giving advice on how to improve it than there are people willing to step up and do the job. Without actually contributing improvements, advice is about as helpful as Sunday morning quarterbacking from the armchair in the living room. There are plenty of good, winning plays already, just not nearly enough players.

-----
Charlie | cyberdash

I think readers who follow the links in the reply to my comment will see my point well made.

The 'read documentation' link is very difficult to find on the Event module page (and why wouldn't the documentation be *on* the event module page). The documentation isn't clear. The 'read documentation' link is also very spotty - my experience is that many of them lead to a generic documentation page, and not module documentation at all.

The link labled 'advanced search option' leads us to a search page, but does not link to any information about advanced search or anything at all, just a set of completely unrelated search results for 'test'. Perhaps there was a point to this link, but it completely eludes me.

I would also point out that I tested the search engibe to generate my example in my original comment. That was yesterday. Unless some overnight coding miracle was performed, I can say that it displays utterly no improvement whatsoever.

The problem is that there are not people to do the sort of massive rewriting and reorganization of the documentation you describe on drupal.org.

This is totally untrue. There are people. They just chose to create Drupal 5 instead.

In all seriousness - until documentation is considered a *part* of coding on Drupal, until the document writers are part of the project team and not just an add-on, until additions and modules are not admitted without documentation, the proplem will persist, as people write code without documentation.

I will also note that the problems of abandoned modules, etc., were not addressed at all.

Sorry. If you click the "advanced search" option on any search return page, you'll see that you can filter the results. That's much improved over the search engine that was available about a year ago.

That link in the event module information for documentation in the example I provided is also a new strategy which has been implemented in the last year, and there was massive effort to provide individual module documentation but not enough people to provide it all. So there are always some improvements being made, but there could always be more. I don't think anyone would disagree that documentation could be improved (I keep saying that :-).

But I disagree with your idea that the people that "chose to create Drupal 5 instead" are the solution for improvements in documentation. There are also people that find the documentation wanting, perceive the need and where the improvements are needed, but do not contribute to the project. It's often the case that those people have an expectation that someone should do it or should have done so already.

But that's not how CBPP works. It's the nature of CBPP that you must have people available who have chosen to do a particular task, not just people already working on the project. And it's not that simple to say this must be done and then the task gets accomplished by someone else. There are so many things to do, and most things get done because someone with a particular interest or motivation takes it on. In fact, I always liked Laura Scott's analogy that an open source project is like a herd of cats. The overall goal of the project is to facilitate those cats in achieving their goals, to strive to provide the environment in which people can make contributions. That makes CBPP work best.

Now, look at where documentation fits within the goals of most coders who are interested in making the software work in a particular way. Sometimes this is because of a particular personal interest (e.g., they like to code, see an interesting problem to solve, or in the case of Drupal, want a particular function or feature on their personal website). Sometimes it's because a client is paying a consultant for some coding. Code is contributed to the project with the hope that it will get picked up by another coder and further developed or someone will provide assistance in bug fixing. They contribute because it is a method of collaboration on software production, not mainly because of altruistic motives of making the world free from proprietary software or providing a public good (although I've seen that there is some satisfaction from that; it's just rarely the primary motivation and more of a side benefit). Certainly there could be some advantages for coders for increased participation in the project by providing some documentation (that's why Drupal has been known for such a well-documented API, although it looks like it needs to be updated at the moment), but often coders feel that the time they have to spend is better spent on coding. And a client's not going to pay for user documentation unless they need it themselves (which they generally don't because the consultant has set up the software for them).

So if you want to get coders to improve documentation, you have to make a strong case for why it is important to them and motivate them, not simply point out what is wrong with the documentation. And if you start seeing what people's motivations are, you'll see that's very, very difficult to do. And if you get involved in the project, you'll find that people are very busy, and you gain ethos for your advice by making contributions first. As the saying goes, "talk is cheap; code is golden." People value contributions because that's where the work gets done. People often follow the lead of other contributors and will help out because they are making things happen.

It all comes down to this. New software users of a project need to understand that CBPP happens because people perceive a problem and contribute by providing fixes and improvements, not just because someone perceives a problem. There is plenty of that going on already and more than enough to do. But if the new user doesn't have the time or inclination themselves, then a new software user should understand why no one else did either. This is the point of my original post. Open source projects need and want people interested in contributing documentation. So if you feel this strongly that documentation needs to be improved, rather than providing more advice, go jump in and help out. Every little bit helps, and I'm sure that the documentation team would be more than happy to provide some guidance on where they are currently focusing their efforts. I know that they would love to have your help :-)

-----
Charlie | cyberdash