Latest revision as of 17:44, 3 March 2019

Other languages:

Semantic Search

The aim of this page is not to fully explain KDE's Semantic Search technology and every detail, but to give a short overview, some examples, share the vision behind it and link to relevant information on the web.

Baloo is the next generation of semantic search

From KDE Applications 4.13 onwards, the 'Baloo' file indexing and file search framework replaces Nepomuk. Read details on the changes for Applications 4.13 here. Semantic Search no longer uses a single, big database, but separate, specialized databases for each type of data. The new search databases are in $HOME/.local/share/baloo. If you upgraded to KDE Applications 4.13 from an earlier KDE release, you can delete $KDEHOME/share/apps/nepomuk.

Kurze Erklärung

As the Glossary mentions, Semantic Search is about classification, organisation and presentation of data. It is not an application, but a component which can be used by developers within applications.

Try out in Dolphin

For example, the Dolphin file manager makes use of Search. In KDE Applications versions prior to 4.13 Semantic Search must be enabled from System settings -> Desktop search. The information sidebar of Dolphin (Control -> Panels -> Information, or press F11) presents information extracted by Search about the selected file, and also allows you to assign tags, ratings and comments to files. This information is then stored and indexed by Search. You can then search for metadata using the navigation bar in Dolphin. Click to Find, or press Ctrl+F, and search for file names or file contents.

Funktionalitäten

KDE's Semantic Search offers several 'layers' of functionality to applications. The first and most simple of those is manual tagging, rating and commenting, as used in Dolphin. This helps you to find your files faster, but is also a lot of work.

To make finding files containing text easier, Search offers a second functionality: indexing the text of files. You can find files by entering some words which you know are in there, or just (part of) their title.

The third layer is a very complex one, and the reason why the underlying technology, Nepomuk, was conceived as a research project of several companies and universities in the European union. This is where you will find difficult words like 'semantic desktop' and 'ontologies'. Basically, it is about context and relationships.

Indexing files

Search does not index every file on the hard drive. Its default configuration in most Linux distributions excludes some common patterns for backup files and configuration directories. You can change this in System Settings -> Desktop Search. Add folders to be excluded. If you want to turn off indexing of files entirely, just add your home folder there.

In System Settings you can also control whether Search indexes files on removable media such as USB drives and CD-ROMs. In KDE Applications 4.13 this is not available, removable media are not indexed. Future versions are planned to reintroduce this functionality.

Beispiele

Let me try to explain what Semantic Search offers using two examples. These features are not available fully yet - the base is there but application developers need to integrate this in their applications.

Verwandtschaft

Angenommen, du hast von einem Freund vor zwei Wochen ein Foto bekommen. Du hast es irgendwo auf deinem Rechner gespeichert. Wie findest du nun diese Datei? Wenn du dich nicht an den Ort erinnerst, hast du Pech.

Now Semantic Search aims to help you. You know this file came from that friend of yours, your computer does not know. Search, however, can remember this relationship. Searching on the name of your friend will therefore pop up the photo!

Another potential relationship is between a web page you copied text from and the document you pasted it into, or two images showing the same car. Such relations can sometimes be extracted from the files themselves (you could analyze photos and see who or what is on there) or supplied by the applications involved (as in the above email example). This part of Search is still under heavy development, and needs integration in applications, so you can expect it to take a few more years to really shine.

All in all, this part of Semantic Search is about making search smart. Think about how Google tries to be smart with your searches: when you search for a hotel and a city name, it shows above the website results a google map showing hotels in the city you mentioned! It might even suggest a better name in case you made a spelling mistake. Google also tries to put the most relevant information on top of the list of results, using complex calculations on relationships (links) between websites. Semantic Search will be able to offer such smart results and order them on relevancy using relationship information.

Kontext

These relationships can not only help you while searching for files, but also have an influence on applications and what information they present. Note that this way of using Search is still more a vision than reality! Many of the components are in place, but it is not yet integrated in applications and the desktop as a whole.

Es folgt ein Beispiel, das dir durch den Kontext hilft, effizienter zu arbeiten.

Angenommen, du arbeitest an der Nachbearbeitung einiger Notizen, die du in einer Besprechung geschrieben hast. Das Telefon klingelt und jemand fragt dich nach einer Datei mit Preisen, und ob du sie für einen Kunden bearbeiten könntest. Nach ein paar weiteren Unterbrechungen ist dein Desktop voller Dateien und Fenster...

Es wäre schön, das alles ein wenig besser zu organisieren, oder?

Enter 'activities'. These have been introduced in Plasma, and currently offer different 'desktops'. They are a bit like virtual desktops, except that the desktop itself changes, not the set of applications. Different widgets, background, things like that. Of course, since Plasma 4.3, each virtual desktop can have its own activity, bringing the two in sync.

Wenn Applikationen und der Desktop sich der Aktivitäten bewusst wären, könnte man eine Aktivität für jede Aufgabe erstellen, an der man regelmäßig arbeitet. Wenn man also oft eine Tabellenkalkulation mit Preisen editieren muss, erstellt man hierfür eine Aktivität: einen oder mehrere Folder View-Widgets auf dem Desktop erstellen, einen Taschenrechner hinzufügen und ein Aufgaben-Widget, um das im Auge zu behalten, was man noch bearbeiten muss. Vielleicht sogar ein E-Mail-Widget, das die E-Mails anzeigt, die Fragen zu diesen Preislisten beinhalten.

Sobald jemand eine Frage zu diesen Preisen stellt, wechselt man zu dieser Aktivität, und startet das Tabellenkalkulationsprogramm. Dieses hat ebenfalls Zugriff auf die Aktivität, und zeigt dir die aktuellsten Preislisten, und nicht die Inventurliste aus einer anderen Aktivität. Kopete, das Chat-Programm zeigt den Kollegen an, der alles über Preise weiß, und die Person ist, mit der man immer chattet, wenn man in dieser Aktivität arbeitet.

Wenn man fertig ist, wechsele zurück in eine andere Aktivität, und alle Programme passen ihr Verhalten wiederum dem an, was man tut.

Die Vorzüge eines solchen aktivitätsbasierten Arbeitsprozesses gehen weiter als man auf den ersten Blick meinen könnte. Es hilft nicht nur dabei, Dateien und Kontaktpersonen zu finden, aber hilft auch, die Aufgabe selber zu wechseln. Das menschliche Gehirn ist nicht sonderlich gut darin, mehrere Dinge gleichzeitig zu machen - die meisten Menschen brauchen mehrere Minuten, um nach einem Aufgabenwechsel wieder Fahrt aufzunehmen. Die 'Umgebung' zu ändern, hilft eine Menge dabei, dieses zu beschleunigen, selbst wenn es sich nur um den Bildschirm handelt. Man vergleiche es damit, in Urlaubslaune zu kommen, indem man lediglich einen Koffer packt.

Natürlich ist all das eher relevant für Leute, die im Büro oder zu Hause an ihrem Rechner arbeiten. Für Spieler oder nur gelegentliche Benutzer würden solche Aktivitäten wahrscheinlich nicht sehr hilfreich sein.

The scenario described above is already partially implemented in the Activities but much work is still left.

Frequently Asked Questions

The following is taken from a KDE forums post. Please feel free to add/remove/modify details if you have the time!

What is the Nepomuk Semantic Desktop, and the Strigi Desktop File Indexer?

Nepomuk and Strigi are technologies part of what delivers the abilities of the Semantic Search in KDE. Both are not used directly in the latest generation of KDE's Semantic Search (details), however their successors share much of their code and concepts. Semantic Search provides a way to organize, annotate and build relationships among the data (not only file name and content, but for example which applications used a certain file, or how it is tagged). A number of KDE applications and workspaces use this basic infrastructure to deliver features such as email tagging (KMail) or activity setup (Plasma).

The file indexing allows applications such as Dolphin to search for files based on content, name, or other meta-data (e.g. tags) associated to indexed files. Such an indexer can also index non-text files, such as PDFs, by accessing the meta-data contained in these files (author, publication information, etc.). Some KDE components ship additional "analyzers" for more file types.

Why do we need both Akonadi and Semantic Search? Aren't they doing the same thing?

In short, Akonadi provides a cache of PIM data like calendar items, contacts and email, which is used by applications like KMail and Korganizer but also the calendar build in Plasma. Semantic Search plugs in Akonadi to provide search functionality. How Baloo offers search is actually up to the application. In case of KDE PIM, Xapian is used to provide indexing and search.

How can I disable the semantic desktop?

File indexing can be disabled by adding the users' home folder to the System Settings -> Desktop Search -> Do not search in these locations list. The other functionality is part of the applications that use it and thus can't be disabled without crippling these applications. For example, to not have any search in KMail you'll have to simply remove KMail...

In versions of the KDE Applications before 4.13, Semantic Search would have components running separate from applications. This functionality could be disabled by unchecking Enable Nepomuk File Indexer in the Desktop Search section of System Settings. In case you want to turn off all semantic features, uncheck Enable Nepomuk Semantic Desktop. Notice that this will turn off search in Dolphin as well.

Notice that with the latter option some programs who use Semantic Search for meta-data will offer reduced functionality: for example KMail will not be able to tag mail, or Plasma activities will not offer additional features such as icons, or program data information.

Baloo/Semantic Search is eating 100% CPU! What do I do?

Just wait. Certain files are very hard or even impossible to Index. At the moment, this includes for example text files of over 50 megabyte. When Search finds these, it will try for a fixed time. When it fails, it will try to find out what file is broken and disable indexing it in the future. As it indexes files in batches of about 40, it has to find the problematic file by indexing that bunch in parts: first half/second half, index problematic half in pieces again, until the file is found. This can take up to 30 minutes of heavy cpu usage. Unfortunately, while Baloo will not start to index a new batch of 40 files while on battery power, it continues to determine the broken file while on battery. This behaviour has been fixed in in KDE Applications 4.13.1 (it will stop indexing immediately when the power cord is unplugged) and the time the search for each file can take has been reduced to about 10 minutes. The Semantic Search team is working on improving the indexing tools to handle more difficult files.

Why do I have nepomukservicestub processes even though I've disabled Nepomuk?

It may be a bug. Please file a bug report with a complete description of your problem and the steps to trigger it.

File indexing of PDF/some other file types doesn't work.

PDF indexing is a known issue and it's being tracked in bug #231936. If you have issues with other files, open a bug, preferably adding a sample file that shows the problem.

The program nepomukservicestub crashes at startup.

A large number of fixes for crashes has been fixed for the 4.7.2 release of the KDE Workspaces and Applications. If you encounter more, please file bugs report with detailed instructions on how to reproduce the problem, as sometimes the developers are unable to trigger them in their test setups.

The virtuoso-t process hangs at 100% CPU.

Virtuoso-t is a key component of the old Semantic Search infrastructure and in some occasions the commands sent by the other components end up taking too much time (hence showing the effect of 100% CPU).

Virtuoso is no longer used by Semantic Search starting the Applications 4.13 release.

Sometimes Nepomuk consumes too much RAM.

Many of these problems have been fixed, in other cases however the developers are unable to reproduce the issues correctly. In this case, providing examples and test cases to bug reports increase the chances to get these bugs fixed.

Search accesses the disk too much on startup.

A throttling mechanism implemented in the file indexer, versions after KDE SC 4.8 should no longer have this issue.

My Search database has been corrupted. How do I clean it?

In the extreme case your database is really corrupted and all other attempts have failed, you can delete the $KDEHOME/share/apps/nepomuk directory (where $KDEHOME is usually .kde or .kde4 in your home directory) while Nepomuk is not running. The database will be cleared, but you will also lose existing information such as tags, ratings and comments.

Advanced troubleshooting

Veröffentlichen und Privatsphäre

Eine Sache muss ich noch erwähnen, bevor ich auf andere Informationsquellen hinweise: das Veröffentlichen von Nepomukdaten. Es wäre von Vorteil, wenn man Tags, Bewertungen und Kommentare mit anderen teilen könnte, wenn man ihnen Dateien sendet. Wenn man jedoch eine Kontaktperson mit mehr oder weniger beleidigenden Tags ("langweilig im Bett") belegt hätte, würde man wahrscheinlich nicht wollen, dass dieser Tag mit versendet würde...

Dieser Umstand wird natürlich bedacht und ist ein wichtiger Aspekt in der Forschung um Nepomuk. Zur Zeit ist diese Privatsphäre zusammen mit den technischen Herausforderungen der Grund, dass Nepomukdaten privat sind. Es sei versichert, dass das Nepomukteam alles erdenkliche tut, damit deine Privatsphäre respektiert wird.

External links

The new Search technology (post KDE Applications 4.13):

The old Search technology:

@@ Line 1: / Line 1: @@
 <languages />
-=Nepomuk=
+=Semantic Search=
-Das Ziel dieser Seite ist es nicht, die Nepomuk-Technologie und jedes ihrer Details vollständig zu erklären, sondern einen kurzen Überblick, einige Beispiele zu geben und die Vision, die dahinter steht zu erklären, sowie weiterführende Links zu hilfreichen Informationen im Netz zu geben.
+The aim of this page is not to fully explain KDE's Semantic Search technology and every detail, but to give a short overview, some examples, share the vision behind it and link to relevant information on the web.
+== Baloo is the next generation of semantic search ==
+From KDE Applications 4.13 onwards, the '[https://community.kde.org/Baloo Baloo]' file indexing and file search framework replaces Nepomuk. Read [http://dot.kde.org/2014/02/24/kdes-next-generation-semantic-search details on the changes for Applications 4.13 here]. Semantic Search no longer uses a single, big database, but separate, specialized databases for each type of data. The new search databases are in <tt>$HOME/.local/share/baloo</tt>. If you upgraded to KDE Applications 4.13 from an earlier KDE release, you can delete <tt>$KDEHOME/share/apps/nepomuk</tt>.
 ==Kurze Erklärung==
-Wie das [[Special:myLanguage/Glossary#Nepomuk|Glossar]] sagt, geht es bei Nepomuk um Klassifikation, Organisation und Präsentation von Daten. Es ist ein bestimmtes Programm, sondern eine Komponente, die von Entwicklern innerhalb ihrer Programme genutzt werden kann.
+As the [[Special:myLanguage/Glossary#Nepomuk|Glossary]] mentions, Semantic Search is about classification, organisation and presentation of data. It is not an application, but a component which can be used by developers within applications.
+=== Try out in Dolphin ===
+For example, the [[Special:myLanguage/Dolphin|Dolphin]] file manager makes use of Search. In KDE Applications versions prior to 4.13 Semantic Search must be enabled from <menuchoice>System settings -> Desktop search</menuchoice>. The information sidebar of Dolphin (<menuchoice>Control -> Panels -> Information</menuchoice>, or press <keycap>F11</keycap>) presents information extracted by Search about the selected file, and also allows you to assign tags, ratings and comments to files. This information is then stored and indexed by Search. You can then search for metadata using the navigation bar in Dolphin. Click to <menuchoice>Find</menuchoice>, or press <keycap>Ctrl+F</keycap>, and search for file names or file contents.
+<!-- info about nepomuksearch:/ deleted per its developer: "Actually nepomuksearch:/ is an internal thing and should not be entered by the user." its developer -->
-<span class="mw-translate-fuzzy">
-=== Ausprobieren ===
-Beispielsweise benutzt [[Special:myLanguage/Dolphin|Dolphin]] Nepomuk. Damit das folgende funktioniert muss Nepomuk und Strigi in <menuchoice>Systemeinstellungen -> Desktopsuche</menuchoice> aktiviert sein. Die Seitenleiste ermöglicht es dir, Tags zu vergeben, Dateien mit Bewertungen und Kommentare zu versehen. Diese Informationen werden in Nepomuk gespeichert und durch Strigi indiziert. Du kannst dann nach Tags suchen, indem du das Suchfeld von Dolphin benutzt. Schreibe dazu "nepomuksearch:/" gefolgt von den Suchbegriffen.
-</span>
 ==Funktionalitäten==
-Nepomuk bietet Applikationen verschiedene Funktionsebenen an. Die einfachste und einfachste ist manuelles Taggen, Bewerten und Kommentieren, wie es in Dolphin benutzt wird. Es hilft dir, deine Dateien schneller zu finden, ist aber auch eine Menge Arbeit.
+KDE's Semantic Search offers several 'layers' of functionality to applications. The first and most simple of those is manual tagging, rating and commenting, as used in Dolphin. This helps you to find your files faster, but is also a lot of work.
-<span class="mw-translate-fuzzy">
+To make finding files containing text easier, Search offers a second functionality: indexing the text of files. You can find files by entering some words which you know are in there, or just (part of) their title.
-Um Textdateien einfacher auffindbar zu machen, bietet Nepomuk eine zweite Funktionalität an: Indizierung des Inhaltes von Dateien. Es benutzt dazu eine Technologie namens [[Special:myLanguage/Glossary#Strigi|Strigi]]. Du kannst dadurch Dateien auch finden, indem du nach einzelnen Wörtern suchst, die in der Datei vorkommen, oder nur nach (einem Teil) des Titels.
-</span>
-Die dritte Ebene ist sehr komplex, und der Grund, warum Nepomuk als Forschungsprojekt einiger Firmen und Universitäten der Europäischen Union entworfen wurde. Hier findest du schwierige Begriffe wie "semantischer Desktop" oder "Ontologien". Prinzipiell geht es um Kontext und Verwandtschaften.
+The third layer is a very complex one, and the reason why the underlying technology, Nepomuk, was conceived as a research project of several companies and universities in the European union. This is where you will find difficult words like 'semantic desktop' and 'ontologies'. Basically, it is about context and relationships.
 === Indexing files ===
-Strigi does not index every file on the hard drive. Its default configuration in most Linux distributions excludes some common patterns for backup files and configuration directories, and it only indexes certain directories in your home folder. You can change this in <menuchoice>System Settings -> Desktop Search -> Desktop index folders -> Customize index folders… -> Strigi Index Folders</menuchoice>.
+Search does not index every file on the hard drive. Its default configuration in most Linux distributions excludes some common patterns for backup files and configuration directories. You can change this in <menuchoice>System Settings -> Desktop Search</menuchoice>. Add folders to be excluded. If you want to turn off indexing of files entirely, just add your home folder there.
-Note that Strigi as of KDE 4.7 does not follow symbolic links ({{bug|208602}}), even if you select folders under the symbolic link for indexing ({{bug|287593}}) for index. (A symbolic link is a file that "points" to another file or directory; Dolphin displays symbolic links in ''italic'' .) You must find the path to the actual directory (in '''Dolphin''', select the file, choose <menuchoice>Properties -> General -> Points to</menuchoice>) and tell Strigi to index that.
+In '''System Settings''' you can also control whether Search indexes files on removable media such as USB drives and CD-ROMs. In KDE Applications 4.13 this is not available, removable media are not indexed. Future versions are planned to reintroduce this functionality.
-In '''System Settings''' you can also control whether Strigi indexes files on removable media such as USB drives and CD-ROMs.
 ==Beispiele==
-Lass mich versuchen, anhand von zwei Beispielen zu erklären, was Nepomuk anbietet.
+Let me try to explain what Semantic Search offers using two examples. These features are not available fully yet - the base is there but application developers need to integrate this in their applications.
 ===Verwandtschaft===
@@ Line 40: / Line 41: @@
 Angenommen, du hast von einem Freund vor zwei Wochen ein Foto bekommen. Du hast es irgendwo auf deinem Rechner gespeichert. Wie findest du nun diese Datei? Wenn du dich nicht an den Ort erinnerst, hast du Pech.
-Nepomuk will dir dabei helfen. Du weißt, dass diese Datei von eben jenem Freund kam, dein Rechner weiß dies jedoch nicht. Nepomuk aber kann sich an diesen Zusammenhang erinnern. In dem du nach dem Namen deines Freundes suchst, wird das Bild also erscheinen!
+Now Semantic Search aims to help you. You know this file came from that friend of yours, your computer does not know. Search, however, can remember this relationship. Searching on the name of your friend will therefore pop up the photo!
-Eine weitere mögliche Verwandtschaft gibt es zwischen einer Internetseite, von der du Text kopiert hast und dem Dokument, in das du es eingefügt hast, oder zwischen zwei Bildern, die dasselbe Auto zeigen. Solche Relationen können manchmal aus den Dateien selber gezogen werden (man könnte Fotos analysieren und schauen, wer oder was darauf zu sehen ist) oder von den benutzten Programmen bereitgestellt werden (wie im obigen E-Mail-Beispiel). Dieser Teil Nepomuks befindet sich immer noch in der Entwicklungsphase und benötigt die Integration in Programmen, so dass man noch einige Jahre warten muss, bis es wirklich funktioniert.
+Another potential relationship is between a web page you copied text from and the document you pasted it into, or two images showing the same car. Such relations can sometimes be extracted from the files themselves (you could analyze photos and see who or what is on there) or supplied by the applications involved (as in the above email example). This part of Search is still under heavy development, and needs integration in applications, so you can expect it to take a few more years to really shine.
-Alles in allem geht es in diesem Teil Nepomuks darum, die Suche zu verbessern. Man denke nur daran, wie Google versucht, Suchanfragen intelligent zu verarbeiten. Wenn man ein Hotel und einen Städtenamen sucht, zeigt die Ergebnisseite einen Stadtplan, der Hotels in der angefragten Stadt anzeigt. Es wird sogar die korrekte Schreibweise angeboten, falls man sich vertippt hat. Google versucht auch, die relevantesten Informationen an den Anfang der Liste zu setzen. Nepomuk wird solche intelligenten Ergebnisse bieten und sie nach Relevanz sortieren können, indem diese Verwandtschaftsinformationen benutzt werden.
+All in all, this part of Semantic Search is about making search smart. Think about how Google tries to be smart with your searches: when you search for a hotel and a city name, it shows above the website results a google map showing hotels in the city you mentioned! It might even suggest a better name in case you made a spelling mistake. Google also tries to put the most relevant information on top of the list of results, using complex calculations on relationships (links) between websites. Semantic Search will be able to offer such smart results and order them on relevancy using relationship information.
 ===Kontext===
-Diese Relationen können nicht nur helfen, wenn man nach Dateien sucht, sondern können auch einen Einfluss auf Programme haben und welche Informationen sie darstellen. Denke daran, dass dieser Teil Nepomuks noch eher eine Vision als Realität ist. Viele Komponenten existieren bereits, sind aber noch nicht in den Programmen und den Desktop insgesamt integriert.
+These relationships can not only help you while searching for files, but also have an influence on applications and what information they present. Note that this way of using Search is still more a vision than reality! Many of the components are in place, but it is not yet integrated in applications and the desktop as a whole.
 Es folgt ein Beispiel, das dir durch den Kontext hilft, effizienter zu arbeiten.
@@ Line 56: / Line 57: @@
 Es wäre schön, das alles ein wenig besser zu organisieren, oder?
-Schau dir '[[Special:myLanguage/Glossary#Activities|Aktivitäten]]' an. Diese wurden in [[Plasma_(de)|Plasma]] eingeführt, und bieten derzeit verschiedene 'Desktops' an. Sie ähneln teils den virtuellen Desktops, außer, dass sich der Desktop selber ändert, nicht die geöffneten Programme. Verschiedene Widgets, ein anderer Hintergrund, und ähnliche Dinge. Natürlich kann seit KDE 4.3 jeder virtuelle Desktop seine eigene Aktivität haben, indem beide miteinander abgeglichen werden.
+Enter '[[Special:myLanguage/Glossary#Activities|activities]]'. These have been introduced in [[Special:myLanguage/Plasma|Plasma]], and currently offer different 'desktops'. They are a bit like virtual desktops, except that the desktop itself changes, not the set of applications. Different widgets, background, things like that. Of course, since Plasma 4.3, each virtual desktop can have its own activity, bringing the two in sync.
 Wenn Applikationen und der Desktop sich der Aktivitäten bewusst wären, könnte man eine Aktivität für jede Aufgabe erstellen, an der man regelmäßig arbeitet. Wenn man also oft eine Tabellenkalkulation mit Preisen editieren muss, erstellt man hierfür eine Aktivität: einen oder mehrere Folder View-Widgets auf dem Desktop erstellen, einen Taschenrechner hinzufügen und ein Aufgaben-Widget, um das im Auge zu behalten, was man noch bearbeiten muss. Vielleicht sogar ein E-Mail-Widget, das die E-Mails anzeigt, die Fragen zu diesen Preislisten beinhalten.
@@ Line 68: / Line 69: @@
 Natürlich ist all das eher relevant für Leute, die im Büro oder zu Hause an ihrem Rechner arbeiten. Für Spieler oder nur gelegentliche Benutzer würden solche Aktivitäten wahrscheinlich nicht sehr hilfreich sein.
-Bedenke, dass das oben beschriebene Szenario noch Jahre in der Zukunft liegt. Ein großer Teil der Infrastruktur dafür existiert in KDE bereits, vieles bleibt jedoch noch zu tun.
+The scenario described above is already partially implemented in the Activities but much work is still left.
 ==Frequently Asked Questions==
@@ Line 74: / Line 75: @@
 The following is taken from a [http://forum.kde.org/viewtopic.php?f=154&t=97098&p=204592 KDE forums] post. Please feel free to add/remove/modify details if you have the time!
-'''Q. What is the Nepomuk Semantic Desktop, and the Strigi Desktop File Indexer?'''
+;What is the Nepomuk Semantic Desktop, and the Strigi Desktop File Indexer?
-A. The Nepomuk Semantic Desktop is the foundation of the all the other modules of the Nepomuk infrastructure. It provides a way to organize, annotate and build relationships among the data (not only file name and content, but for example which applications used a certain file, or how it is tagged). A number of KDE applications and workspaces use this basic infrastructure to deliver features such as email tagging ([[Special:myLanguage/KMail|KMail]]) or activity setup (Plasma).
+: Nepomuk and Strigi are technologies part of what delivers the abilities of the Semantic Search in KDE. Both are not used directly in the latest generation of KDE's Semantic Search ([http://dot.kde.org/2014/02/24/kdes-next-generation-semantic-search details]), however their successors share much of their code and concepts. Semantic Search provides a way to organize, annotate and build relationships among the data (not only file name and content, but for example which applications used a certain file, or how it is tagged). A number of KDE applications and workspaces use this basic infrastructure to deliver features such as email tagging ([[Special:myLanguage/KMail|KMail]]) or activity setup (Plasma).
-On the other hand, the Strigi Desktop File Indexer is a system to index files so that they can be added to the main Nepomuk repository, a convenient way to use them within Nepomuk without adding any file manually. Also, applications such as [[Special:myLanguage/Dolphin|Dolphin]] can then search for files basing on content, name, or other meta-data (e.g. tags) associated to indexed files. Such an indexer can also index non-text files, such as PDFs, by accessing the meta-data contained in these files (author, publication information, etc.). Some KDE components ship additional "analyzers" for more file types. Nepomuk can be fully functional without the use of the File Indexer, which is an additional (and optional) component.
+:The file indexing allows applications such as [[Special:myLanguage/Dolphin|Dolphin]] to search for files based on content, name, or other meta-data (e.g. tags) associated to indexed files. Such an indexer can also index non-text files, such as PDFs, by accessing the meta-data contained in these files (author, publication information, etc.). Some KDE components ship additional "analyzers" for more file types.
-'''Q. How can I tell if Strigi has indexed a file?'''
+; Why do we need both Akonadi and Semantic Search?  Aren't they doing the same thing?
-A. In Dolphin, select the file. If the Information panel displays "Created at" and "Has hash", then the file was indexed by Strigi.
+: In short, Akonadi provides a cache of PIM data like calendar items, contacts and email, which is used by applications like KMail and Korganizer but also the calendar build in Plasma. Semantic Search plugs in Akonadi to provide search functionality. How Baloo offers search is actually up to the application. In case of KDE PIM, Xapian is used to provide indexing and search.
-'''Q. How can I disable the semantic desktop?'''
+;How can I disable the semantic desktop?
-A. Most of the times, the easiest way is to disable file indexing, which is usually, among the Nepomuk components, the heavier in resource usage (although many optimizations have been included in the 4.7 release which reduce resource usage). This is done by unchecking <menuchoice>Enable Strigi Desktop File Indexer</menuchoice> in the [[Special:myLanguage/System_Settings/Search_Desktop|Desktop Search]] section of [[Special:myLanguage/System Settings|System Settings]]. In case you want to turn off all semantic features, uncheck <menuchoice>Enable Nepomuk semantic desktop</menuchoice>. Notice that this will turn off search in [[Special:myLanguage/Dolphin|Dolphin]] as well.
+: File indexing can be disabled by adding the users' home folder to the <menuchoice>System Settings -> Desktop Search -> Do not search in these locations list</menuchoice>. The other functionality is part of the applications that use it and thus can't be disabled without crippling these applications. For example, to not have any search in KMail you'll have to simply remove KMail...
-Notice that with the latter option some programs who use Nepomuk for meta-data will offer reduced functionality: for example [[Special:myLanguage/KMail|KMail]] will not be able to tag mail, or Plasma activities will not offer additional features such as icons, or program data information.
+In versions of the KDE Applications before 4.13, Semantic Search would have components running separate from applications. This functionality could be disabled by unchecking <menuchoice>Enable Nepomuk File Indexer</menuchoice> in the [[Special:myLanguage/System_Settings/Search_Desktop|Desktop Search]] section of [[Special:myLanguage/System Settings|System Settings]]. In case you want to turn off all semantic features, uncheck <menuchoice>Enable Nepomuk Semantic Desktop</menuchoice>. Notice that this will turn off search in [[Special:myLanguage/Dolphin|Dolphin]] as well.
-'''Q. Why do I have nepomukservicestub processes even though I've disabled Nepomuk?'''
+:Notice that with the latter option some programs who use Semantic Search for meta-data will offer reduced functionality: for example [[Special:myLanguage/KMail|KMail]] will not be able to tag mail, or Plasma activities will not offer additional features such as icons, or program data information.
-A. It may be a bug. Please file a [http://bugs.kde.org bug report] with a complete description of your problem and the steps to trigger it.
+;Baloo/Semantic Search is eating 100% CPU! What do I do?
-'''Q. File indexing of PDF/some other file types doesn't work.'''
+:Just wait. Certain files are very hard or even impossible to Index. At the moment, this includes for example text files of over 50 megabyte. When Search finds these, it will try for a fixed time. When it fails, it will try to find out what file is broken and disable indexing it in the future. As it indexes files in batches of about 40, it has to find the problematic file by indexing that bunch in parts: first half/second half, index problematic half in pieces again, until the file is found. This can take up to 30 minutes of heavy cpu usage. Unfortunately, while Baloo will not start to index a new batch of 40 files while on battery power, it continues to determine the broken file while on battery. This behaviour has been fixed in in KDE Applications 4.13.1 (it will stop indexing immediately when the power cord is unplugged) and the time the search for each file can take has been reduced to about 10 minutes. The Semantic Search team is working on improving the indexing tools to handle more difficult files.
-A. PDF indexing is a known issue and it's being tracked in {{bug|231936}}. If you have issues with other files, open a bug, preferably adding a sample file that shows the problem.
+;Why do I have nepomukservicestub processes even though I've disabled Nepomuk?
-'''Q. The program nepomukservicestub crashes at startup.'''
+:It may be a bug. Please file a [http://bugs.kde.org bug report] with a complete description of your problem and the steps to trigger it.
-A. A large number of fixes for crashes has been fixed for the 4.7.2 release of the KDE Workspaces and Applications. If you encounter more, please file bugs report with detailed instructions on how to reproduce the problem, as sometimes the developers are unable to trigger them in their test setups.
+;File indexing of PDF/some other file types doesn't work.
-'''Q. The virtuoso-t process hangs at 100% CPU.'''
+:PDF indexing is a known issue and it's being tracked in {{bug|231936}}. If you have issues with other files, open a bug, preferably adding a sample file that shows the problem.
-A. Virtuoso-t is a key component of the Nepomuk infrastructure and in some occasions the commands sent by the other components end up taking too much time (hence showing the effect of 100% CPU). Sebastian Trüg (the lead developer of Nepomuk) has fixed most of these problems in 4.7.1 or newer.
+;The program nepomukservicestub crashes at startup.
-'''Q. Sometimes Nepomuk consumes too much RAM.'''
+:A large number of fixes for crashes has been fixed for the 4.7.2 release of the KDE Workspaces and Applications. If you encounter more, please file bugs report with detailed instructions on how to reproduce the problem, as sometimes the developers are unable to trigger them in their test setups.
-A. Many of these problems have been fixed, in other cases however the developers are unable to reproduce the issues correctly. In this case, providing examples and test cases to [http://bugs.kde.org/ bug reports] increase the chances to get these bugs fixed.
+;The virtuoso-t process hangs at 100% CPU.
-'''Q. Nepomuk re-indexes files at startup.'''
+:Virtuoso-t is a key component of the old Semantic Search infrastructure and in some occasions the commands sent by the other components end up taking too much time (hence showing the effect of 100% CPU).
-A. This bug has been fixed in 4.7.0 versions. Now Nepomuk just "scans" for changes, without indexing anything.
+Virtuoso is no longer used by Semantic Search starting the Applications 4.13 release.
-'''Q. Nepomuk accesses the disk too much on startup.'''
+;Sometimes Nepomuk consumes too much RAM.
-A. In 4.7 and newer this problem has been lessened thanks to a throttling mechanism implemented in the file indexer.
+:Many of these problems have been fixed, in other cases however the developers are unable to reproduce the issues correctly. In this case, providing examples and test cases to [http://bugs.kde.org/ bug reports] increase the chances to get these bugs fixed.
-'''Q. My Nepomuk database has been corrupted. How do I clean it?'''
+;Search accesses the disk too much on startup.:
-A. In the extreme case your database is really corrupted and all other attempts have failed, you can delete the <tt>$KDEHOME/share/apps/nepomuk</tt> directory (where <tt>$KDEHOME</tt> is usually <tt>.kde</tt> or <tt>.kde4</tt>) while Nepomuk is not running. The database will be cleared, but you will also lose existing information such as tags, ratings and comments.
+:A throttling mechanism implemented in the file indexer, versions after KDE SC 4.8 should no longer have this issue.
-== Advanced troubleshooting ==
+;My Search database has been corrupted. How do I clean it?
-If Nepomuk has trouble indexing a file, in a terminal try running
+:In the extreme case your database is really corrupted and all other attempts have failed, you can delete the <tt>$KDEHOME/share/apps/nepomuk</tt> directory (where <tt>$KDEHOME</tt> is usually <tt>.kde</tt> or <tt>.kde4</tt> in your home directory) while Nepomuk is not running. The database will be cleared, but you will also lose existing information such as tags, ratings and comments.
-{{Input|1=nepomukindexer ''/path/to/file''}}
-and see if there's any useful output. You can compare the output to a similar file that is successfully indexed.
-{{Input|1=xmlindexer ''/path/to/file'' > /tmp/test.xml}}
+== Advanced troubleshooting ==
-generates an XML representation of some of the information extracted from a file. You can view this in an XML viewer, such as your browser, and again compare it to a similar file. (<code>xmlindexer</code> may be in a different and optional package in your Linux distribution; for example in Ubuntu it is in the '''strigi-utils''' package.)
-[http://kdeatopensuse.wordpress.com/2011/11/09/debugging-nepomukvirtuosos-cpu-usage/ This blog post] explains how to turn on debugging output from the [[Glossary#Nepomuk|Nepomuk]] service. It also shows how to debug the CPU usage of '''Nepomuk''' and its storage backend.
 ==Veröffentlichen und Privatsphäre==
@@ Line 139: / Line 135: @@
 Dieser Umstand wird natürlich bedacht  und ist ein wichtiger Aspekt in der Forschung um Nepomuk. Zur Zeit ist diese Privatsphäre zusammen mit den technischen Herausforderungen der Grund, dass Nepomukdaten privat sind. Es sei versichert, dass das Nepomukteam alles erdenkliche tut, damit deine Privatsphäre respektiert wird.
-<span class="mw-translate-fuzzy">
+== External links ==
-:''Mehr Informationen'':
-::[http://de.wikipedia.org/wiki/Semantischer_Desktop Wikipedia - Semantischer Desktop]
-::[http://de.wikipedia.org/wiki/NEPOMUK_%28Framework%29  Wikipedia - NEPOMUK Framework]
-::[http://nepomuk.semanticdesktop.org/xwiki/bin/view/Main1/ NEPOMUK website]
-::[http://nepomuk.kde.org/discover/user NEPOMUK KDE site]
-::[http://dot.kde.org/2009/12/10/exploring-new-nepomuk-features-mandriva-linux-2010 article explaining Nepomuk on the DOT]
-</span>
+The new Search technology (post KDE Applications 4.13):
+* [http://dot.kde.org/2014/02/24/kdes-next-generation-semantic-search user information article on the dot]
+* [http://community.kde.org/Baloo Developer information on community.kde.org]
 * [http://en.wikipedia.org/wiki/Semantic_desktop Wikipedia - Semantic Desktop]
+The old Search technology:
+* [http://techbase.kde.org/Projects/Nepomuk Nepomuk pages for developers on KDE TechBase]
+* [http://nepomuk.kde.org/discover/user NEPOMUK KDE site]
 * [http://en.wikipedia.org/wiki/NEPOMUK_(framework)  Wikipedia - NEPOMUK Framework]
-* [http://nepomuk.semanticdesktop.org/xwiki/bin/view/Main1/ NEPOMUK website] ''inactive?''
+* [http://nepomuk.semanticdesktop.org/nepomuk/ NEPOMUK website]
-* [http://nepomuk.kde.org/discover/user NEPOMUK KDE site]
 * [http://dot.kde.org/2009/12/10/exploring-new-nepomuk-features-mandriva-linux-2010 article explaining Nepomuk on the DOT]
+* [http://kdenepomukmanual.wordpress.com Getting started user manual]
-[[Category:System_(de)]]
+[[Category:System/de]]