View source for User:Claus chr/DPL/Test

== Testing DPL ==

{{Input|1=<nowiki>{{Search|\[\[Special:myLanguage/Applications}}</nowiki>}}

{{Search|\[\[Special:myLanguage/Applications}}

This only finds new style links, and conversely find links to subpages as well. Better use the LinksTo template

{{Input|1=<nowiki>{{LinksTo|Applications}}</nowiki>}}

{{LinksTo|Applications}}

<!--
{{Input|1=<nowiki>
<DPL>
  namespace=Main |User
  nottitleregexp = .*(/..(.|-..)?|_[(].*[)])$
  titlematch = A%|B%|C%|D%|E%|F%|G%|H%|I%|J%
  include = *
  includematch = #\[[Cc]ategory:#
  includemaxlength = 0
  resultsheader = The pages in the translation containing a category statement (A-J): %TOTALPAGES%\n
  format = ,\n* [[%PAGE%|%TITLE%]],,
</DPL>
</nowiki>}}

<DPL>
  namespace=Main
  nottitleregexp = .*(/..(.|-..)?|_[(].*[)])$
  titlematch = A%|B%|C%|D%|E%|F%|G%|H%|I%|J%
  include = *
  includematch = #\[[Cc]ategory:#
  includemaxlength = 0
  resultsheader = The pages in the translation containing a category statement (A-J):%TOTALPAGES%\n
  format = ,\n* [[%PAGE%|%TITLE%]],,
</DPL>
----
<DPL>
  namespace=Main
  nottitleregexp = .*((/[a-z][a-z](.|-..)?)|([ _][(][a-z][a-z][)]))$
  titlematch = K%
  include = *
  includematch = #\[[Cc]ategory:#
  includemaxlength = 0
  resultsheader = The pages in the translation containing a category statement (K):%TOTALPAGES%\n
  format = ,\n* [[%PAGE%|%TITLE%]],,
</DPL>
----
<DPL>
  namespace=Main
  nottitleregexp = .*(/..(.|-..)?|_[(].*[)])$
  titlematch = L%|M%|N%|O%|P%|Q%|R%|S%|T%|U%|V%|W%|X%|Y%|Z%
  include = *
  includematch = #\[[Cc]ategory:#
  includemaxlength = 0
  resultsheader = The pages in the translation containing a category statement (L-Z):%TOTALPAGES%\n
  format = ,\n* [[%PAGE%|%TITLE%]],,
</DPL>
->>
<!--
<DPL>
  namespace=Main|User
  nottitleregexp = .*(/..(-..)?|_[(].*[)])$
  include = *
  includematch = #\[\[([Ss]pecial\:[mM]y[lL]anguage/)?Getting[_ ]Help#
  includemaxlength = 0
  resultsheader = The pages in the translation system linking to Getting Help are:\n
  format = ,\n* [[%PAGE%|%TITLE%]],,
  resultsheader=<pre><nowiki>
  resultsfooter=</nowiki></pre>
</DPL>

Somehow this is broken. Without the namespace clause we get

<DPL>
  nottitleregexp = .*(/..(-..)?|_[(].*[)])$
  include = *
  includematch = #\[\[([Ss]pecial\:[mM]y[lL]anguage/)?Getting[_ ]Help#
  includemaxlength = 0
  resultsheader = The pages in the translation system linking to Getting Help are:\n
  format = ,\n* [[%PAGE%|%TITLE%]],,
</DPL>

which excludes some pages in the main namespace(!) and includes on in the User namespace!? Explicitely specifying namespace main gives this.

<DPL>
  namespace = Main
  nottitleregexp = .*(/..(-..)?|_[(].*[)])$
  include = *
  includematch = #\[\[([Ss]pecial\:[mM]y[lL]anguage/)?Getting[_ ]Help#
  includemaxlength = 0
  resultsheader = The pages in the translation system linking to Getting Help are:\n
  format = ,\n* [[%PAGE%|%TITLE%]],,
</DPL>

That seems reasonable, but weren't main supposed to be used by default? Finally, specifying namespace User gives us all three user pages:

<DPL>
  namespace=User
  nottitleregexp = .*(/..(-..)?|_[(].*[)])$
  include = *
  includematch = #\[\[([Ss]pecial\:[mM]y[lL]anguage/)?Getting[_ ]Help#
  includemaxlength = 0
  resultsheader = The pages in the translation system linking to Getting Help are:\n
  format = ,\n* [[%PAGE%|%TITLE%]],,
</DPL>

=== All English pages linking to a given page (template version) ===

The lesson here seems to be that, at least when include is involved, we can't rely on DPL handling more than one namespace at a time. This calls for a template: <nowiki>{{LinksTo|Getting[_ ]Help}} gives</nowiki>

{{LinksTo|1=Getting[_ ]Help}}

So, to recap, the problem is this: We would like to find all pages that links to a given page (the target). The <code>What links here</code> wiki page does not work well with links adapted to the translation extension, i.e. links of the form <nowiki>[[Special:myLanguage/</nowiki>'''''target page'''''<nowiki>]]</nowiki>, which means that almost none of our links would be found.

Instead we have to use DPL to find those pages, but even here we have to be careful. The obvious search using the <code>linksto</code> clause doesn't work either, probably for the same reason that <code>What Links Here</code> failed. The solution is to search the content of every page for the occurrence of a link to the target page. The problem here is, that DPL has to write the entire content of a page into the page where the search occurs, then search the text, then filter out the text again (that's the <code>includemaxlength=0</code> part) and leave only links to the pages that contain a link to the target page.

For some reason, these kinds of query cause some confusion to DPL. Specifying no namespace should result in the main namespace beeing searched, but as we saw above, not every matching page in main was found, and oddly one User: page was found. Specifying both namespaces in one search seems to work better, but still misses a User: page. The solution seems to be one search for each namespace: that seems to find everything with one exception: The start page [[Welcome to KDE UserBase]] seems to be outside of namespaces, and is never found. Let's hope it is one of a kind.

This method of searching has been implemented as a template <nowiki>{{LinksTo|</nowiki>'''''target page'''''<nowiki>}}</nowiki> (which can easily be modified to include more namespaces in the search).
Just to be clear: We are searching for the actual occurence of a link in the texts of pages. The search finds both Special:myLanguage links and old-style links. It tries to take every known variation into account (Special:, or special:, and mylanguage, Mylanguage, myLanguage, or MyLanguage). One variation we have to deal with by hand is, that spaces can be written either as a normal space chracter or as an underscore. Therefore we should either make to searches if the target name contains a space, or we could write <code>[_ ]</code> wherever a space occurs in the name, as in <code><nowiki>{{LinksTo|Getting[_ ]Help}}</nowiki></code>. Also note, that case is significant: <code><nowiki>{{LinksTo|getting[_ ]help}}</nowiki></code> yields

{{LinksTo|getting[_ ]help}}
----
We just get two error messages (one for each search), since DPL gives an error message whenever a text search finds no match on any page.

=== The template ===

The code of the first half of the template is here:
{{Input|<nowiki>
{{#dpl:
| namespace = Main
| nottitleregexp = .*(/..(-..)?{{!}}_[(].*[)])$
| include = *
| includematch = @\[\[([Ss]pecial\:[mM]y[lL]anguage/)?{{{1}}}(\]{{!}}\{{!}}{{!}}#)@
| includemaxlength = 0
| format = ,\n* [[%PAGE%|%TITLE%]],,
}}
</nowiki>}}

The second half is the same, except that <code><nowiki>| namespace = Main</nowiki></code> is replaced by <code><nowiki>| namespace = User</nowiki></code>.

I couldn't make normal DPL tags work in the template, but fortunately the <code><nowiki>{{#dpl</nowiki></code> parser function does work. To add more namespaces just add a copy of the first half of the template to then end, and in the copy replace <code><nowiki>| namespace = Main</nowiki></code> by <code><nowiki>| namespace = Whatever</nowiki></code>. From the DPL documentation you might think that to search the Main namespace, you could leave out the <code>namespace</code> caluse altogether: That is not a good idea &mdash; in this context, it gets DPL confused so that it misses pages!

The <code>nottitleregexp</code> clause filters out any page whoes path ends in <tt>/xx</tt>, <tt>/xx-xx</tt>, or <tt>_(x..)</tt>, ie. it filters out all translated pages, both old and new. Since the pipe character has special meaning in a template, it has to be entered as <nowiki>{{!}}</nowiki>.

The <code>includematch</code> clause is a perl regexpr, that matches text of the form <tt><nowiki>[[Special:myLanguage/</nowiki>''page path''</tt> followed by either a '<nowiki>]</nowiki>', a '<nowiki>|</nowiki>', or a '#' character. We take into account that the S, the M, and the L of <tt>Special:MyLanguage</tt> are sometimes capitalized, and sometimes not.

The <code>includemaxlength = 0</code> is to prevent DPL from entering (parts of) the content of the matched pages into the page containing the query.

=== Testing the template ===

<code><nowiki>{{LinksTo|User:Claus[_ ]chr}}</nowiki></code>{{LinksTo|User:Claus[_ ]chr}}

Problem nr. 1. We also find links to subpages of the target (doh!), but that should be simple to fix. (The error message just means that there were no linking pages in the Main namespace.)

<code><nowiki>{{LinksTo|Amarok}}</nowiki></code>{{LinksTo|Amarok}}

Yes! There must be thousands of pages linking to some subpage of Amarok. They are obviously not found. Now can I find this link to [[Talk:Translation Workflow]]?

<code><nowiki>{{TestLinksTo|Talk:Translation[_ ]Workflow}}</nowiki></code>{{TestLinksTo|Talk:Translation[_ ]Workflow}}

This page is not found, but it is found if the same query is performed in another page! I guess, I should have expected that, given the way these queries are performed. Otherwise the page would have to include itself, which could lead to problems.

<code><nowiki>{{LinksTo|Amarok/Manual}}</nowiki></code>{{LinksTo|Amarok/Manual}}

So we can find subpages &mdash; good thing too!

=== Addendum ===

* 30. June 2011: Added search of the Talk namespace plus a line of text to make the template document its use.
* 18. July 2011: It seems this doesn't work! We do not find all the pages containing a given link. It seems that some resource is exceeded, and that only part of the pages are actually searched: 
Consider the following two searches:

{{Input|1=<nowiki>
<DPL>
  nottitlematch = %/__
  namespace = Main
  include = *
  includematch = /[Ss]eamless/
  includemaxlength = 0
  resultsheader = Nontranslated pages containing the string "seamless"
  format = ,\n* [[%PAGE%|%TITLE%]]\n,,
</DPL>
</nowiki>}}

<DPL>
  nottitlematch = %/__
  namespace = Main
  include = *
  includematch = /[Ss]eamless/
  includemaxlength = 0
  resultsheader = Nontranslated pages containing the string "seamless"
  format = ,\n* [[%PAGE%|%TITLE%]]\n,,
</DPL>

{{Input|1=<nowiki>
<DPL>
  nottitlematch = %/__|%/__-__|% (%|%/___
  namespace = Main
  include = *
  includematch = /[Ss]eamless/
  includemaxlength = 0
  resultsheader = Nontranslated pages containing the string "seamless"
  format = ,\n* [[%PAGE%|%TITLE%]]\n,,
</DPL>
</nowiki>}}

<DPL>
  nottitlematch = %/__|%/__-__|% (%|%/___
  namespace = Main
  include = *
  includematch = /[Ss]eamless/
  includemaxlength = 0
  resultsheader = Nontranslated pages containing the string "seamless"
  format = ,\n* [[%PAGE%|%TITLE%]]\n,,
</DPL>

The first search excludes most translated pages, namely those ending on <tt>/la</tt> with ''la'' a two letter language code; the second search is identical except that it excludes all translated pages including remaining old style translations. You would expect the latter search to find fewer matches not more since it searches among fewer pages. On the other hand, if some capacity is exceeded by the search then the first search would run out of space first and so find fewer hits.
-->