The Invisible Web Weblog

Journal of “Online Information Review” is going to publish a special issue on “Evaluating Web Search Engines”. The topics include: retrieval effectiveness, reliability of search results, language handling, index sizes and overlap comparisons, query log analysis, user behaviour, results presentation, user surveys, user guidance in the search process, ability of search engines to deal with different query types, index and/or results freshness, diversity of results, search features comparison, influence of search engine optimization (SEO) on results quality.

I just came back from a wonderful visit of Esfahan, a remarkable historic city in the middle of the county. In 25 May 2009, the main library of the Esfahan University held a one day seminar about the visible and invisible web. I was one of the invited speakers and I delivered a one hour presentation entitled “The Visibility and Invisibility of Information on the Web”. It was a brief summary of my studies on the invisible web concept over the last eight years. At the end of the seminar, I also took part in a panel to answer the participants’ questions. I can summarize my main message to the seminar simply as follows: “ … in my theory of information visibility on the web, I basically put the web user in the focal point of the definition of the invisible web and also consider the search context’s effects in the level of information visibility for each search session. This approach is considerably different with the common definition of this concept which is only focused on search engines’...

"Search Interfaces on the Web: Querying and Characterizing" is a PhD thesis by Denis Shestakov, in the Department of Information Technology at the University of Turku in Finland. This study is an example of the recent technical-oriented studies on the Deep Web. More details about this work including the abstract and the full text are available here .

The latest progress in web-based search industry is always one of my favourite areas of study. If you look through the related literature you can see various techniques claiming better search results. However, I think what we really need is new search engines which are able to make sense of what we ask them. This approach is called differently in various sources such as "Search 2.0" or "Semantic Search". Hakia compares this approach with the traditional indexing methods and concludes that: "Semantic search technology has many advantages compared to statistical (popularity) methods, especially when statistical data is not available such as in dynamic content and long-tail queries". I don't know how successful they have been in implementation of this new approach but for more information please have a look at this address .

Dr. Matthew Koll in 1999 at a presentation entitled "Major Trends and Issues in the Information Industry" has used the interesting metaphor of Needle in a Haystack for searching the Web. He has grouped different search scenarios into 12 categories as follows: a known needle in a known haystack, a known needle in an unknown haystack, an unknown needle in an unknown haystack, any needle in a haystack, the sharpest needle in a haystack, most of the sharpest needles in a haystack All the needles in a haystack, affirmation of no needles in the haystack, things like needles in any haystack, let me know whenever a new needle shows up, where are the haystacks? and finally needles, haystacks -- whatever. I'd like this metaphor and I think we can map different search scenarios into it very easily which can be helpful to develop efficient search strategies.

I have been informed about a website with links to a wide range of online resources which are useful for end users to search deeper layers of the Web. " 99 Resources to Research & Mine the Invisible Web " is a good example of Web-based directories which can facilitate Web searching for everyone.

This is my first summer after the completion of my PhD with less work and more free time. However, I am not totally on leave and do some educational activities. For example, I ran a one-day workshop at Astan Quds library on Tuesday 31 July. The workshop's topic was about my main research interest, the invisible web and information seeking behaviour on the web. I explored the differences between the literature-based definition of the invisible web and what I mean by this phenomenon based on my PhD research. More than thirty members of the library staff participated in the workshop. Some of the comments that I received about the model of cognitive invisibility (published in 2006 in JDOC vol. 62, issue 5) were very useful to view the model from the perspective of some experienced librarians. I tried to explain the practical implications of the model in theory, policy and practice. I also had a speech today for a group of the librarians who work in the cultural centres. It was about th...

Last week I went to Isfahan to deliver a presentation at the conference of Challenges of Information Science at Isfahan University on last Wednesday. The title of my presentation was "What is Grounded Theory and What is Its Application?" I talked about the nature of Grounded Theory and how we can use this methodology in our research. Also, yesterday I had another presentation in Tehran about "The Role of the Invisible Web on Information Accessibility of Biomedical Databases" at The School of Public Health and Institute of Public Health Research which is the oldest and largest center of health studies in Iran. A group of academic staff, researchers and PhD students of the school attended the presentation.

During these days I was busy with running different workshops and participation in some research panels. Of course, meanwhile I was involved in my routine teaching and research activities at my university as well. 1. I had another workshop about the invisible web at Irandoc on last Wednesday and Thursday (2 and 3 May 2007). Twenty participants, who were mainly from different academic libraries in Tehran and other cities, took part in the workshop and I explored different aspects of the invisible web over this two-day workshop. 2. I also participated in a panel yesterday (6 May) about the role of Information Literacy in the development of Iran at Tehran International Book Fair . 3. I had a workshop about advanced method of web searching for students in the department of educational technology. 4. I will have a two-day workshop about the basics of Information Literacy at Irandoc in the near future.

I ran my workshop about the invisible web at Irandoc and in my opinion it was very successful. Twenty participants took part in the workshop during two days and they showed their enthusiasm about the topic by their active participation in the discussions and practices. Most of them were academic librarians and mangers of libraries from different parts of Iran. It was very encouraging for me that they participated in the discussion so enthusiastically. I will run the same workshop at Irandoc for another group in the next week.

I am going to run a two-day workshop in 18 and 19 April at IRANDOC centre. The workshop's title is "An Introduction to the Invisible Web Phenomenon and the Theory of Information Invisibility on the Web". For more information about the registration in the workshop please contact the education section of IRANDOC at: edu @ irandoc.ac.ir

Dirk Lewandowski from the Department of Information Science, Heinrich-Heine-University, Düsseldorf and Philipp Mayrthe at the Institute of Library and Information Science, Humboldt-University, Berlin, discussed about the Academic Invisible Web (AIW). They define AIW as "... all databases and collections relevant to academia but not searchable by the general-purpose internet search engines." For more deatiles look attheir paper at this address .

This is just a quick link to the " Digital Curation Centre Public Survey ".

Jakob Nielsen has discussed the issue of "weblog usability" in a succinct and efficient way. In his website he has a webpage “ Weblog Usability: The Top Ten Design Mistakes ” in which he identifies the following points as top ten mistakes that bloggers should avoid to enhance the usability of their weblogs. In terms of Nielsen’s webpage these mistakes are: "No Author Biographies, (2) No Author Photo, (3) Nondescript Posting Titles, (4) Links Don't Say Where They Go, (5) Classic Hits are Buried, (6) The Calendar is the Only Navigation, (7) Irregular Publishing Frequency, (8) Mixing Topics, (9) Forgetting That You Write for Your Future Boss, and (10) Having a Domain Name Owned by a Weblog Service." You can read the whole story about the usability of weblogs in Nielsen's website . He also has written “ Top Ten Web Design Mistakes ”.

I know Google is the most popular search engine and almost everyone uses it to search the Web. Of course, there is no doubt about the efficiency of Google in providing Web users with an easy and quick answer to any kind of questions. However, the quickest answer is not necessarily the best answer. Sometimes it might be the best but not always. The problem is you never know what else exist on the Web that could be retrieved by the search tool for you. Anyway, what I am trying to say is “do not use only one search engine all the time”. It might be worth trying alternative tools occasionally. Who knows it might work better. There are other search tools which are very good. Vivisimo is one of them.

Developing a proper search strategy is a pivotal skill to succeed a search procedure. In fact, this expertise is considered as one of the key abilities for information literate people and it requires some skills including a reasonable level of familiarity with existing search tools. Noodletools.com is a helpful website that covers a great ground in this issue. For example, in the page entitled “ Information Literacy: Search Strategies ” you will find an interesting table that provides you with many useful search tools based on the type of your information needs. When you employ an appropriate search tool you will notice how suitability of the employed search tool with your information needs can make a difference in your search results.

This is just a quick link to Google Librarian and Google’s Newsletter for Librarians , and an online paper about how Google index and rank web-based resources . That is it for now.

According to the result of a recent study carried out by Antonio Gulli and Alessio Signorini the size of the indexable web is estimated to be more than 11.5 billion pages. They reported this result at the special interest tracks and posters of the 14th international conference on World Wide Web in Chiba, Japan, May 2005. The full document is available here .

This is just a quick link to a meta-search engine called ProFusion powered by Intelliseek company. ProFusion provides its users with deeper search realm over the web. I’ll write more about this search tools later on because this is one the search facilities related to the invisible web and interestingly when you search the invisible web term in Google, ProFusion is one of the first retrieved items. Actually, I should write about it much earlier, but I don’t know for whatever reason I put it off. Anyway, I have further details about ProFusion but I would like you visit it first and if you like try its search ability , then I’ll write my ideas.

Broken links, dead links or Linkrot are those URLs which are not available anymore and lead to nowhere. This problem might happen for many reasons. For example, they probably have moved form their original place or have been replaced by new information. This can be a very important issue for everybody who uses the web. In fact, broken links are one of the major parts of the opaque web as well. If you have a favourite links collection on your own web browser you probably have already noticed that how quickly these links getting out of date and you need to keep your list up-to-date constantly.