A survey of users of the SciDev.Net website has provided us with important feedback about the value of the information we present, and the ways in which it is utilised. Both will feed directly into plans for the development of the website.
More than half of SciDev.Net's regular users say that the website has allowed them to increase their professional knowledge, and just under half (49 per cent) of our users in developing countries say that it has helped to increase their awareness of the role of science and technology in economic and social development.
Two-thirds say that the most useful aspect of the website is the way that it keeps them up-to-date with relevant news, while more than half (54 per cent) value it for bringing important issues to their attention.
About 80 per cent of the users are involved in issues related to science and technology in developing countries that involve decision-making in some way; for a quarter of these, policy-related work forms the main part of their job.
And the most popular dossier is that focusing on biodiversity — which is consulted regularly by 29 per cent of registered users (and 31 per cent in developing countries) — followed closely by the dossiers on the ethics of research and on indigenous knowledge.
These are some of the results to emerge from an electronic survey of the users of the SciDev.Net website that was carried out in July 2004 to help us evaluate the use of the site, with the goals of using this information to develop its content.
All those who are registered with the site - at the time, around 12,000 — as well as unregistered visitors, were asked to complete an electronic questionnaire before entering the site. The questionnaire contained 45 questions; ten of these required completion, and the remainder were left voluntary.
In total, 1,530 replies were received. Although those who chose to reply to the questionnaire were obviously self-selected, the distribution of the respondents — for example, in terms of geographical region, or profession — closely reflects the distribution of the overall pool of registrants.
In addition, there was in general a close correlation between the proportion of responses received from users in developed and developing countries (in the survey, 42 per cent and 58 per cent, respectively), and the overall distribution of our registered users.
Both of these factors suggest that even though our survey was not particularly scientific, its results appears to be relatively robust in statistical terms, at least insofar as making relative judgements about different aspects and characteristics of the website.
The information that the results contain will therefore be used as an invaluable input into our plans for developing new features on the website, such as such as tailoring the weekly email alert (perhaps by region, or by topic), supplying 'breaking' news alerts, and improving both the content and appearance of dossiers and quick guides.
During the next two years, we will continue to monitor and assess our services and expand our evaluation techniques, perhaps to include focus groups in selected countries within the developing world. Details will be posted on the site as these plans evolve.
Meanwhile we would like to thank those who took the time to complete the survey, whose results will, as mentioned above, provide invaluable input into our plans for developing the website. We are especially grateful to users who said their Internet access was slow and unreliable (11 per cent of respondents in the developing world) and who still took time to send us their thoughts and comments.
Detailed responses to the users survey are as follows:
- The most common professions of users of the website are science researchers (19.7 per cent of respondents) and university teachers (15 per cent). The next category is science journalists (7.5 per cent).
- Overall, more than half of the respondents to the survey (58 per cent) were from developing countries, with the highest figures from individual countries being India (9.7 per cent) and Brazil (8.5 per cent).
- More than two-thirds of respondents reported that they visit the website at least once a week, with 22 per cent saying that they visit the website several times a week.
- Almost half respondents agreed that they found the website either "excellent" (44 per cent) or "indispensable" (6.2 per cent). A further 30.5 per cent described it as either "good" or "useful".
- Asked to indicate the subject covered by the website that they found of most interest, 65 per cent of the respondents identified science communication, 63 per cent identified science publishing, and 61 per cent identified biodiversity.
- In response to a question about what users valued most about SciDev.Net, 67 per cent agreed that "it keeps me up-to-date with relevant news", 54 per cent that "it brings my attention to important issues", 48 per cent that "it provides valuable background information", and 47 per cent that "it is a good source of relevant reports and contacts".
Asked about the individual sections of the website, the news section came out top, with 75 per cent of respondents from the developing countries — and 82 per cent from the developed countries — agreeing with the statement that this section was either "very valuable" or "valuable".
The proportion of respondents in developing countries applying the same description to other parts of the website were as follows: opinions (69 per cent); features (68 per cent); editorials (65 per cent); e-guide to science communication (59 per cent); quick guides (58 per cent); and dossiers (55 per cent).
Use of information
In response to a separate question about the way that the information on the website is used; 53 per cent of all respondents agreed that "it has allowed me to expand my professional knowledge"; 49 per cent of respondents in developing countries (and 47 per cent overall) agreed that "it has increased my awareness of the importance of science and technology in social and economic development"; 40 per cent from developing countries (and 35 per cent overall) agreed that "it has helped me to make up my mind on critical issues relating to the role of science and technology in development"; and 33 per cent agreed that "it has helped me to increase the awareness of others of the importance of science and technology in social and economic development".
Given that we have a particular interest in providing information that relates to policy issues and decision-making around science and technology in developing countries, we were keen to find out more about our users' involvement in such issues.
In response, 40 per cent said that they were actively involved in developing policy, engaging in policy-related discussions, or researching on policy-related issues, although it was not the main part of their job; 20 per cent said that the above activities were the main part of their job; and 20 per cent said they were not engaged in such issues professionally, but were engaged in policy activities outside their professional work.
To help guide our thoughts about how to increase our impact with 'policy-shapers', we asked those completing the section on dossiers a couple of questions relating to SciDev.Net's 'policy briefs'. 37 per cent said it was either "vital" or "important" that policy briefs were written by experts in their fields (compared to 6 per cent who said it was "not that important", or "not important at all"). Thirty-five per cent said that 'pdfs' of policy briefs would be either "useful" or "very useful".
A further question asked about ways in which those visiting the website use our news items and other material. Nearly half (44 per cent) of respondents said they had emailed a news item to a friend, and 39 per cent had discussed our news items with colleagues and friends. Forty-four per cent had printed out a news item for their own use. And 38 per cent had used the links at the bottom of each news item to visits the websites of relevant organisations, and / or to consult related items appearing on other websites.
Similarly 29 per cent had printed a feature for their own use, 26 per cent had emailed a feature to a friend, and 21 per cent had used the links to relevant organisations, websites and / or related stories. Twenty-three per cent had printed out an editorial for their own use, 22 per cent had discussed editorials with colleagues and friends, and 18 per cent had emailed an editorial to a friend. And 20 per cent had used a news item in their own writing or on a website, while a similar proportion had used a news item for use by others (such as in a classroom setting).
A separate question asked about the use of material contained in our dossiers. About half of the respondent replied to this question (which was among the voluntary questions). Of these, 32 per cent said that they had shared information with colleagues, 13 per cent had used dossiers to assist with their research or to help back up an argument (14 per cent), and 8 per cent had used the information in a classroom setting.
Asked about the length of news articles on the website, 63 per cent thought that this was "about right"; only 6 per cent felt that they were too short, and 2 per cent that they were too long. Similarly, 68 per cent of respondents thought that the language in which news stories are written was "about right"; 2 per cent thought that they were too technical, and 2 per cent that they were too simple.
A separate question asked about opinion articles. Fifty-six per cent of respondents said that they read these at least once a month, and 45 per cent said that the number of opinion articles was "about right", compared to 5 per cent who said that there were not enough, and 3 per cent who said that there were too many.
As for the range of opinions expressed in such articles — another question that did not require a response — 37 per cent of all respondents said that the range of opinions expressed was about right, and 12 per cent said that the opinion articles were not sufficiently controversial (compared to 5 per cent who said that they were too biased and lacked sufficient reflection).
Concerning feature articles, 55 per cent of respondents said that they read these at least once a month. Twenty-four per cent of respondents said that they would like to see more features on the website, 31 per cent said that they would like to see "about the same" — and only one per cent who would like to see fewer).
More on dossiers
Several questions asked about dossiers. Overall, those consulted most frequently by respondents were as follows: biodiversity (29 per cent); ethics of research (24 per cent); indigenous knowledge (23 per cent); intellectual property (22 per cent), and climate change (20 per cent).
As for the contents of individual dossiers, those considered to be either "valuable" or "very valuable" were as follows: news (46 per cent); features (44 per cent); key documents (42 per cent), policy briefs (41 per cent); opinion articles (41 per cent); links (40 per cent); spotlights (33 per cent); events (33 per cent and training 27 per cent).
Asked which new elements should be added to dossiers 38 per cent identified regional or national case studies and 25 per cent identified country-specific fact sheets.
Finally, users of the site were asked a number of questions about their use of the notices section, The most regularly consulted sections within this section were events (35 per cent) and announcements (32 per cent), followed by grants (23 per cent) and jobs (16 per cent).
Asked whether they had ever responded to notices, 31 per cent said that they had found out more information on an event or meeting; 27 per cent had emailed a notice to a colleague; 8 per cent had attended a listed event, meeting or workshop; 6 per cent had applied for a listed grant; and 5 per cent had applied for a listed job.