Welcome to TiddlyWiki created by Jeremy Ruston; Copyright © 2004-2007 Jeremy Ruston, Copyright © 2007-2011 UnaMesa Association
[img[http://alpage.inria.fr/statgram/frdep/I/alpage.png]] http://alpage.inria.fr/alpc.en.html
> Alpage project team develops and maintains a full linguistic processing chain for French (try our online demo) . This chain is based on [[DyALog|http://alpage.inria.fr/dyalog.en.html]], [[FRMG|http://alpage.inria.fr/catalogue.en.html#frmg]], [[Lefff|http://alpage.inria.fr/~sagot/lefff.html]], [[SxPipe|http://alpage.inria.fr/catalogue.fr.html#sxpipe]].
<<<
* ''~DyALog'' is an environment to compile and run logic programs and natural language tabular parsers for various grammatical formalisms (DCGs, TAGs, TIGs, RCGs).
* ''frmg'' is a a French grammar generated from a MetaGrammar and compiled with DyALog.
* ''Lefff'' (Lexique des Formes Fléchies du Français / Lexicon of French inflected forms) is a large-scale morphological and syntactic lexicon for French, distributed under the LGPL-LR free software license (Lesser General Public License For Linguistic Resources).
* ''sxpipe'' is a pre-parsing processing chain that handles segmentation and tokenization, spelling error correction, and named entities regognition. It is designed to transform in a robust way raw corpus to DAG of lexical entries.
<<<
[img[http://alpage.inria.fr/statgram/frdep/I/alpage.png]] http://alpage.inria.fr/statgram/frdep/fr_stat_dep_malt.html
!Parsing French using the MaltParser
Combination de [[MElt|MElt (ALPAGE Linguistic Workbench)]] et de [[MaltParser]] pour l'analyse de texte brut en français.
;Input :
: texte brut, ''utf-8''
;Output :
:the 10 usual [[CoNLL|CoNLL data format]] columns, plus an extra column for word cluster ids (between the 6th and 7th usual CoNLL columns)
<<<
''Prerequisites''
* [[MaltParser|http://www.maltparser.org/]] version __1.3.1__, developed by Johan Hall, Jens Nilsson and Joakim Nivre at Växjö University and Uppsala University, Sweden. (Note it won't work with later malt versions)
* the [[MElt|https://gforge.inria.fr/projects/lingwb]] tagger (downloadable [[here|https://gforge.inria.fr/frs/download.php/27240/melt-0.6.tar.gz]]), developed by Pascal Denis & Benoît Sagot, ([[Coupling an annotated corpus and a morphosyntactic lexicon for state-of-the-art POS tagging with less human effort|http://pauillac.inria.fr/~pdenis/papers/paclic09.pdf]]. In Proceedings of PACLIC 2009, Hong Kong, China).
* download and unzip the [[BONSAI|http://alpage.inria.fr/statgram/frdep/Code/bonsai_v3.2.tgz]] v3.2 archive new!, to get preprocessing code, and malt model and settings (best Malt model according to benchmark : uses predicted POS, predicted lemmas, predicted morpho features, and unsupervised word clusters). Note preprocessing code requires:
** perl and python >2.5
** [[python-cjson|http://pypi.python.org/pypi/python-cjson]], to install with : python setup.py install
* Set the MALT_DIR variable to your local path to Malt 1.3.1
* Set the BONSAI variable to your local path to BONSAI v3.2
Parsing command
The following command will preprocess and parse a raw UTF-8 text file INFILE into INFILE.outmalt :
{{{$BONSAI/bin/bonsai_malt_parse_rawtext.sh [-n] INFILE}}}
Use -n option if your text is already tokenized
Note : The output format is (almost) CoNLL : the 10 usual CoNLL columns, plus an extra column for word cluster ids (between the 6th and 7th usual CoNLL columns).
Note : newlines in input text are systematically interpreted as sentence frontiers.
<<<
Tagset used by : <<list filter [tag[tags:CC]]>>
The current tagset used by [[MElt|MElt (ALPAGE Linguistic Workbench)]] is as follows (Crabbé & Candito, 2008)
Crabbé, Candito 08 : [[Expériences d'analyse syntaxique du français|http://hal-univ-diderot.archives-ouvertes.fr/docs/00/34/10/93/PDF/crabbecandi-taln2008-final.pdf]], in Actes de TALN 2008 (Traitement automatique des langues naturelles), Avignon.
Optimisation pour l'analyse syntaxique à 29 tags, des 34 sous-catégories du [[FTB|French TreeBank POS Tags]]
|ADJ | adjective |
|ADJWH | interrogative adjective |
|ADV | adverb |
|ADVWH | interrogative adverb |
|CC | coordination conjunction |
|CLO | object clitic pronoun |
|CLR | reflexive clitic pronoun |
|CLS | subject clitic pronoun |
|CS | subordination conjunction |
|DET | determiner |
|DETWH | interrogative determiner |
|ET | foreign word |
|I | interjection |
|NC | common noun |
|NPP | proper noun |
|P | preposition |
|P+D | preposition+determiner amalgam |
|P+PRO | prepositon+pronoun amalgam |
|PONCT | punctuation mark |
|PREF | prefix |
|PRO | full pronoun |
|PROREL | relative pronoun |
|PROWH | interrogative pronoun |
|V | indicative or conditional verb form |
|VIMP | imperative verb form |
|VINF | infinitive verb form |
|VPP | past participle |
|VPR | present participle |
|VS | subjunctive verb form |
analyse syntaxique de surface
http://nextens.uvt.nl/depparse-wiki/DataFormat
<<<
Data adheres to the following rules:
* Data files contain sentences separated by a blank line.
* A sentence consists of one or tokens, each one starting on a new line.
* A token consists of ten fields described in the table below. Fields are separated by a single tab character. Space/blank characters are not allowed in within fields
* All data files will contains these ten fields, although only the ID, FORM, CPOSTAG, POSTAG, HEAD and DEPREL columns are guaranteed to contain non-dummy (i.e. non-underscore) values for all languages.
* Data files are UTF-8 encoded (Unicode). If you think this will be a problem, have a look [[here|http://nextens.uvt.nl/depparse-wiki/UnicodeHelp]].
|Field number: |Field name: |Description: |h
|1 |ID |Token counter, starting at 1 for each new sentence. |
|2 |FORM |Word form or punctuation symbol. |
|3 |LEMMA |Lemma or stem (depending on particular data set) of word form, or an underscore if not available. |
|4 |CPOSTAG |Coarse-grained part-of-speech tag, where tagset depends on the language. |
|5 |POSTAG |Fine-grained part-of-speech tag, where the tagset depends on the language, or identical to the coarse-grained part-of-speech tag if not available. |
|6 |FEATS |Unordered set of syntactic and/or morphological features (depending on the particular language), separated by a vertical bar (|), or an underscore if not available. |
|7 |HEAD |Head of the current token, which is either a value of ID or zero ('0'). Note that depending on the original treebank annotation, there may be multiple tokens with an ID of zero. |
|8 |DEPREL |Dependency relation to the HEAD. The set of dependency relations depends on the particular language. Note that depending on the original treebank annotation, the dependency relation may be meaningful or simply 'ROOT'. |
|9 |PHEAD |Projective head of current token, which is either a value of ID or zero ('0'), or an underscore if not available. Note that depending on the original treebank annotation, there may be multiple tokens an with ID of zero. The dependency structure resulting from the PHEAD column is guaranteed to be projective (but is not available for all languages), whereas the structures resulting from the HEAD column will be non-projective for some sentences of some languages (but is always available). |
|10 |PDEPREL |Dependency relation to the PHEAD, or an underscore if not available. The set of dependency relations depends on the particular language. Note that depending on the original treebank annotation, the dependency relation may be meaningful or simply 'ROOT'. |
<<<
[[Main Page]]
[[POS tagger links]]
<!--{{{-->
<div class='toolbar' macro='toolbar [[ToolbarCommands::EditToolbar]]'</div>
<div class='title' macro='view title'></div>
<div class='editor' macro='edit title'></div>
<div macro='annotations'></div>
<div class='editor' macro='edit text'></div>
<div class='editor' macro='edit tags'></div><div class='editorFooter'><span macro='message views.editor.tagPrompt'></span><span macro='tagChooser excludeLists'></span></div>
<!--}}}-->
/***
|''Name:''|FieldsEditorPlugin|
|''Description:''|//create//, //edit//, //view// and //delete// commands in toolbar <<toolbar fields>>.|
|''Version:''|1.0.2|
|''Date:''|Dec 21,2007|
|''Source:''|http://visualtw.ouvaton.org/VisualTW.html|
|''Author:''|Pascal Collin|
|''License:''|[[BSD open source license|License]]|
|''~CoreVersion:''|2.2.0|
|''Browser:''|Firefox 2.0; InternetExplorer 6.0, others|
!Demo:
On [[homepage|http://visualtw.ouvaton.org/VisualTW.html]], see [[FieldEditor example]]
!Installation:
*import this tiddler from [[homepage|http://visualtw.ouvaton.org/VisualTW.html]] (tagged as systemConfig)
*save and reload
*optionnaly : add the following css text in your StyleSheet : {{{#popup tr.fieldTableRow td {padding:1px 3px 1px 3px;}}}}
!Code
***/
//{{{
config.commands.fields.handlePopup = function(popup,title) {
var tiddler = store.fetchTiddler(title);
if(!tiddler)
return;
var fields = {};
store.forEachField(tiddler,function(tiddler,fieldName,value) {fields[fieldName] = value;},true);
var items = [];
for(var t in fields) {
var editCommand = "<<untiddledCall editFieldDialog "+escape(title)+" "+escape(t)+">>";
var deleteCommand = "<<untiddledCall deleteField "+escape(title)+" "+escape(t)+">>";
var renameCommand = "<<untiddledCall renameField "+escape(title)+" "+escape(t)+">>";
items.push({field: t,value: fields[t], actions: editCommand+renameCommand+deleteCommand});
}
items.sort(function(a,b) {return a.field < b.field ? -1 : (a.field == b.field ? 0 : +1);});
var createNewCommand = "<<untiddledCall createField "+escape(title)+">>";
items.push({field : "", value : "", actions:createNewCommand });
if(items.length > 0)
ListView.create(popup,items,this.listViewTemplate);
else
createTiddlyElement(popup,"div",null,null,this.emptyText);
}
config.commands.fields.listViewTemplate = {
columns: [
{name: 'Field', field: 'field', title: "Field", type: 'String'},
{name: 'Actions', field: 'actions', title: "Actions", type: 'WikiText'},
{name: 'Value', field: 'value', title: "Value", type: 'WikiText'}
],
rowClasses: [
{className: 'fieldTableRow', field: 'actions'}
],
buttons: [ //can't use button for selected then delete, because click on checkbox will hide the popup
]
}
config.macros.untiddledCall = { // when called from listview, tiddler is unset, so we need to pass tiddler as parameter
handler : function(place,macroName,params,wikifier,paramString) {
var macroName = params.shift();
if (macroName) var macro = config.macros[macroName];
var title = params.shift();
if (title) var tiddler = store.getTiddler(unescape(title));
if (macro) macro.handler(place,macroName,params,wikifier,paramString,tiddler);
}
}
config.macros.deleteField = {
handler : function(place,macroName,params,wikifier,paramString,tiddler) {
if(!readOnly && params[0]) {
fieldName = unescape(params[0]);
var btn = createTiddlyButton(place,"delete", "delete "+fieldName,this.onClickDeleteField);
btn.setAttribute("title",tiddler.title);
btn.setAttribute("fieldName", fieldName);
}
},
onClickDeleteField : function() {
var title=this.getAttribute("title");
var fieldName=this.getAttribute("fieldName");
var tiddler = store.getTiddler(title);
if (tiddler && fieldName && confirm("delete field " + fieldName+" from " + title +" tiddler ?")) {
delete tiddler.fields[fieldName];
store.saveTiddler(tiddler.title,tiddler.title,tiddler.text,tiddler.modifier,tiddler.modified,tiddler.tags,tiddler.fields);
story.refreshTiddler(title,"ViewTemplate",true);
}
return false;
}
}
config.macros.createField = {
handler : function(place,macroName,params,wikifier,paramString,tiddler) {
if(!readOnly) {
var btn = createTiddlyButton(place,"create new", "create a new field",this.onClickCreateField);
btn.setAttribute("title",tiddler.title);
}
},
onClickCreateField : function() {
var title=this.getAttribute("title");
var tiddler = store.getTiddler(title);
if (tiddler) {
var fieldName = prompt("Field name","");
if (store.getValue(tiddler,fieldName)) {
window.alert("This field already exists.");
}
else if (fieldName) {
var v = prompt("Field value","");
tiddler.fields[fieldName]=v;
store.saveTiddler(tiddler.title,tiddler.title,tiddler.text,tiddler.modifier,tiddler.modified,tiddler.tags,tiddler.fields);
story.refreshTiddler(title,"ViewTemplate",true);
}
}
return false;
}
}
config.macros.editFieldDialog = {
handler : function(place,macroName,params,wikifier,paramString,tiddler) {
if(!readOnly && params[0]) {
fieldName = unescape(params[0]);
var btn = createTiddlyButton(place,"edit", "edit this field",this.onClickEditFieldDialog);
btn.setAttribute("title",tiddler.title);
btn.setAttribute("fieldName", fieldName);
}
},
onClickEditFieldDialog : function() {
var title=this.getAttribute("title");
var tiddler = store.getTiddler(title);
var fieldName=this.getAttribute("fieldName");
if (tiddler && fieldName) {
var value = tiddler.fields[fieldName];
value = value ? value : "";
var lines = value.match(/\n/mg);
lines = lines ? true : false;
if (!lines || confirm("This field contains more than one line. Only the first line will be kept if you edit it here. Proceed ?")) {
var v = prompt("Field value",value);
tiddler.fields[fieldName]=v;
store.saveTiddler(tiddler.title,tiddler.title,tiddler.text,tiddler.modifier,tiddler.modified,tiddler.tags,tiddler.fields);
story.refreshTiddler(title,"ViewTemplate",true);
}
}
return false;
}
}
config.macros.renameField = {
handler : function(place,macroName,params,wikifier,paramString,tiddler) {
if(!readOnly && params[0]) {
fieldName = unescape(params[0]);
var btn = createTiddlyButton(place,"rename", "rename "+fieldName,this.onClickRenameField);
btn.setAttribute("title",tiddler.title);
btn.setAttribute("fieldName", fieldName);
}
},
onClickRenameField : function() {
var title=this.getAttribute("title");
var fieldName=this.getAttribute("fieldName");
var tiddler = store.getTiddler(title);
if (tiddler && fieldName) {
var newName = prompt("Rename " + fieldName + " as ?", fieldName);
if (newName) {
tiddler.fields[newName]=tiddler.fields[fieldName];
delete tiddler.fields[fieldName];
store.saveTiddler(tiddler.title,tiddler.title,tiddler.text,tiddler.modifier,tiddler.modified,tiddler.tags,tiddler.fields);
story.refreshTiddler(title,"ViewTemplate",true);
}
}
return false;
}
}
config.shadowTiddlers.StyleSheetFieldsEditor = "/*{{{*/\n";
config.shadowTiddlers.StyleSheetFieldsEditor += ".fieldTableRow td {padding : 1px 3px}\n";
config.shadowTiddlers.StyleSheetFieldsEditor += ".fieldTableRow .button {border:0; padding : 0 0.2em}\n";
config.shadowTiddlers.StyleSheetFieldsEditor +="/*}}}*/";
store.addNotification("StyleSheetFieldsEditor", refreshStyles);
//}}}
http://nlp.lsi.upc.edu/freeling/
<<<
The FreeLing package consists of a library providing language analysis services.
FreeLing is designed to be used as an external library from any application requiring this kind of services. Nevertheless, a simple main program is also provided as a basic interface to the library, which enables the user to analyze text files from the command line.
Main services offered by FreeLing library:
* Text tokenization
* Sentence splitting
* Morphological analysis
* Suffix treatment, retokenization of clitic pronouns
* Flexible multiword recognition
* Contraction splitting
* Probabilistic prediction of unkown word categories
* Named entity detection
* Recognition of dates, numbers, ratios, currency, and physical magnitudes (speed, weight, temperature, density, etc.)
* PoS tagging
* Chart-based shallow parsing
* Named entity classification
* WordNet based sense annotation and disambiguation
* Rule-based dependency parsing
* Nominal correference resolution
Currently supported languages are Spanish, Catalan, Galician, Italian, English, Russian, Portuguese, Welsh and Asturian. See the [[user manual|http://nlp.lsi.upc.edu/freeling/doc/userman/html/node5.html]] for more information about which services are available for each language.
<<<
Tagset used by : <<list filter [tag[tags:FTB]]>>
http://www.ims.uni-stuttgart.de/~schmid/french-tagset.html
14 catégories, 34 sous-catégories (optimisé à 29 étiquettes : [[CC Tagset]])
|>|ABR |abreviation |
|>|ADJ |adjective |
|>|ADV |adverb |
|>|DET ||
| |DET:ART |article |
| |DET:POS |possessive pronoun (ma, ta, ...) |
|>|INT |interjection |
|>|KON |conjunction |
|>|NAM |proper name |
|>|NOM |noun |
|>|NUM |numeral |
|>|PRO |pronoun |
| |PRO:DEM |demonstrative pronoun |
| |PRO:IND |indefinite pronoun |
| |PRO:PER |personal pronoun |
| |PRO:POS |possessive pronoun (mien, tien, ...) |
| |PRO:REL |relative pronoun |
|>|PRP |preposition |
| |PRP:det |preposition plus article (au,du,aux,des) |
|>|PUN |punctuation |
| |PUN:cit |punctuation citation |
|>|SENT |sentence tag |
|>|SYM |symbol |
|>|VER ||
| |VER:cond |verb conditional |
| |VER:futu |verb futur |
| |VER:impe |verb imperative |
| |VER:impf |verb imperfect |
| |VER:infi |verb infinitive |
| |VER:pper |verb past participle |
| |VER:ppre |verb present participle |
| |VER:pres |verb present |
| |VER:simp |verb simple past |
| |VER:subi |verb subjunctive imperfect |
| |VER:subp |verb subjunctive present |
<<search>><<permaview>><<toggleSideBarTB right hide>>
http://code.google.com/p/hunpos/
;Input :
:tokenized text : one token per line, empty lines are sentence separators.
;Output/Trainning data :
:adding the predicted part-of-speech tag to the end of each line, separated by a TAB
:{{{FORM POS}}}
;Tag sets :
:EN: [[Wall Street Journal (WSJ) Tagset]]
:HU: the Szeged Corpus (http://www.inf.u-szeged.hu/projectdirs/hlt/index_en.html)
> Hunpos is an open source reimplementation of TnT, the well known part-of-speech tagger by Thorsten Brants.
<<<
Features
* Free and open source, even for commercial use.
* For languages with more complex morphologies, HMM tagging could be quite competitive with the current generation of learning algorithms applying e.g. SVM and CRF methods. A major advantage is that the training/tagging cycle is orders of magnitude faster than in more complex models.
* Precision of tagging on unknown and unseen words was a major priority for us during the development of hunpos.
* Works smoothly with large tag sets. For example in Hungarian, as in other highly inflecting languages, it is important to preserve detailed morphological information in the POS tags in order to provide useful clues for higher level processing tasks. This leads to a significantly larger tagset than is common in English (744 tags here as opposed to the 36 standardly used in Treebank work), but does not degrade training and tagging performance. Although it would make the training process of non-generative models computationally expensive.
* Effortless integration of knowledge from morphological analyzers/dictionaries into best path calculation.
* Contextualized lexical probabilities with a context window of any size. Unlike traditional HMM models, HunPos estimates emission (lexical) probabilities based on the current tag and previous tags as well.
* Hunpos has been implemented in OCaml, a high-level language which supports a succinct, well-maintainable coding style. OCaml has a high-performance compiler that produces native code with speed comparable to C/C++ implementations.
<<<
/***
|Name|ImageSizePlugin|
|Source|http://www.TiddlyTools.com/#ImageSizePlugin|
|Version|1.2.3|
|Author|Eric Shulman|
|License|http://www.TiddlyTools.com/#LegalStatements|
|~CoreVersion|2.1|
|Type|plugin|
|Description|adds support for resizing images|
This plugin adds optional syntax to scale an image to a specified width and height and/or interactively resize the image with the mouse.
!!!!!Usage
<<<
The extended image syntax is:
{{{
[img(w+,h+)[...][...]]
}}}
where ''(w,h)'' indicates the desired width and height (in CSS units, e.g., px, em, cm, in, or %). Use ''auto'' (or a blank value) for either dimension to scale that dimension proportionally (i.e., maintain the aspect ratio). You can also calculate a CSS value 'on-the-fly' by using a //javascript expression// enclosed between """{{""" and """}}""". Appending a plus sign (+) to a dimension enables interactive resizing in that dimension (by dragging the mouse inside the image). Use ~SHIFT-click to show the full-sized (un-scaled) image. Use ~CTRL-click to restore the starting size (either scaled or full-sized).
<<<
!!!!!Examples
<<<
{{{
[img(100px+,75px+)[images/meow2.jpg]]
}}}
[img(100px+,75px+)[images/meow2.jpg]]
{{{
[<img(34%+,+)[images/meow.gif]]
[<img(21% ,+)[images/meow.gif]]
[<img(13%+, )[images/meow.gif]]
[<img( 8%+, )[images/meow.gif]]
[<img( 5% , )[images/meow.gif]]
[<img( 3% , )[images/meow.gif]]
[<img( 2% , )[images/meow.gif]]
[img( 1%+,+)[images/meow.gif]]
}}}
[<img(34%+,+)[images/meow.gif]]
[<img(21% ,+)[images/meow.gif]]
[<img(13%+, )[images/meow.gif]]
[<img( 8%+, )[images/meow.gif]]
[<img( 5% , )[images/meow.gif]]
[<img( 3% , )[images/meow.gif]]
[<img( 2% , )[images/meow.gif]]
[img( 1%+,+)[images/meow.gif]]
{{tagClear{
}}}
<<<
!!!!!Revisions
<<<
2011.09.03 [1.2.3] bypass addStretchHandlers() if no '+' suffix is used (i.e., not resizable)
2010.07.24 [1.2.2] moved tip/dragtip text to config.formatterHelpers.imageSize object to enable customization
2009.02.24 [1.2.1] cleanup width/height regexp, use '+' suffix for resizing
2009.02.22 [1.2.0] added stretchable images
2008.01.19 [1.1.0] added evaluated width/height values
2008.01.18 [1.0.1] regexp for "(width,height)" now passes all CSS values to browser for validation
2008.01.17 [1.0.0] initial release
<<<
!!!!!Code
***/
//{{{
version.extensions.ImageSizePlugin= {major: 1, minor: 2, revision: 3, date: new Date(2011,9,3)};
//}}}
//{{{
var f=config.formatters[config.formatters.findByField("name","image")];
f.match="\\[[<>]?[Ii][Mm][Gg](?:\\([^,]*,[^\\)]*\\))?\\[";
f.lookaheadRegExp=/\[([<]?)(>?)[Ii][Mm][Gg](?:\(([^,]*),([^\)]*)\))?\[(?:([^\|\]]+)\|)?([^\[\]\|]+)\](?:\[([^\]]*)\])?\]/mg;
f.handler=function(w) {
this.lookaheadRegExp.lastIndex = w.matchStart;
var lookaheadMatch = this.lookaheadRegExp.exec(w.source)
if(lookaheadMatch && lookaheadMatch.index == w.matchStart) {
var floatLeft=lookaheadMatch[1];
var floatRight=lookaheadMatch[2];
var width=lookaheadMatch[3];
var height=lookaheadMatch[4];
var tooltip=lookaheadMatch[5];
var src=lookaheadMatch[6];
var link=lookaheadMatch[7];
// Simple bracketted link
var e = w.output;
if(link) { // LINKED IMAGE
if (config.formatterHelpers.isExternalLink(link)) {
if (config.macros.attach && config.macros.attach.isAttachment(link)) {
// see [[AttachFilePluginFormatters]]
e = createExternalLink(w.output,link);
e.href=config.macros.attach.getAttachment(link);
e.title = config.macros.attach.linkTooltip + link;
} else
e = createExternalLink(w.output,link);
} else
e = createTiddlyLink(w.output,link,false,null,w.isStatic);
addClass(e,"imageLink");
}
var img = createTiddlyElement(e,"img");
if(floatLeft) img.align="left"; else if(floatRight) img.align="right";
if(width||height) {
var x=width.trim(); var y=height.trim();
var stretchW=(x.substr(x.length-1,1)=='+'); if (stretchW) x=x.substr(0,x.length-1);
var stretchH=(y.substr(y.length-1,1)=='+'); if (stretchH) y=y.substr(0,y.length-1);
if (x.substr(0,2)=="{{")
{ try{x=eval(x.substr(2,x.length-4))} catch(e){displayMessage(e.description||e.toString())} }
if (y.substr(0,2)=="{{")
{ try{y=eval(y.substr(2,y.length-4))} catch(e){displayMessage(e.description||e.toString())} }
img.style.width=x.trim(); img.style.height=y.trim();
if (stretchW||stretchH) config.formatterHelpers.addStretchHandlers(img,stretchW,stretchH);
}
if(tooltip) img.title = tooltip;
// GET IMAGE SOURCE
if (config.macros.attach && config.macros.attach.isAttachment(src))
src=config.macros.attach.getAttachment(src); // see [[AttachFilePluginFormatters]]
else if (config.formatterHelpers.resolvePath) { // see [[ImagePathPlugin]]
if (config.browser.isIE || config.browser.isSafari) {
img.onerror=(function(){
this.src=config.formatterHelpers.resolvePath(this.src,false);
return false;
});
} else
src=config.formatterHelpers.resolvePath(src,true);
}
img.src=src;
w.nextMatch = this.lookaheadRegExp.lastIndex;
}
}
config.formatterHelpers.imageSize={
tip: 'SHIFT-CLICK=show full size, CTRL-CLICK=restore initial size',
dragtip: 'DRAG=stretch/shrink, '
}
config.formatterHelpers.addStretchHandlers=function(e,stretchW,stretchH) {
e.title=((stretchW||stretchH)?this.imageSize.dragtip:'')+this.imageSize.tip;
e.statusMsg='width=%0, height=%1';
e.style.cursor='move';
e.originalW=e.style.width;
e.originalH=e.style.height;
e.minW=Math.max(e.offsetWidth/20,10);
e.minH=Math.max(e.offsetHeight/20,10);
e.stretchW=stretchW;
e.stretchH=stretchH;
e.onmousedown=function(ev) { var ev=ev||window.event;
this.sizing=true;
this.startX=!config.browser.isIE?ev.pageX:(ev.clientX+findScrollX());
this.startY=!config.browser.isIE?ev.pageY:(ev.clientY+findScrollY());
this.startW=this.offsetWidth;
this.startH=this.offsetHeight;
return false;
};
e.onmousemove=function(ev) { var ev=ev||window.event;
if (this.sizing) {
var s=this.style;
var currX=!config.browser.isIE?ev.pageX:(ev.clientX+findScrollX());
var currY=!config.browser.isIE?ev.pageY:(ev.clientY+findScrollY());
var newW=(currX-this.offsetLeft)/(this.startX-this.offsetLeft)*this.startW;
var newH=(currY-this.offsetTop )/(this.startY-this.offsetTop )*this.startH;
if (this.stretchW) s.width =Math.floor(Math.max(newW,this.minW))+'px';
if (this.stretchH) s.height=Math.floor(Math.max(newH,this.minH))+'px';
clearMessage(); displayMessage(this.statusMsg.format([s.width,s.height]));
}
return false;
};
e.onmouseup=function(ev) { var ev=ev||window.event;
if (ev.shiftKey) { this.style.width=this.style.height=''; }
if (ev.ctrlKey) { this.style.width=this.originalW; this.style.height=this.originalH; }
this.sizing=false;
clearMessage();
return false;
};
e.onmouseout=function(ev) { var ev=ev||window.event;
this.sizing=false;
clearMessage();
return false;
};
}
//}}}
/***
|Name|ImportTiddlersPlugin|
|Source|http://www.TiddlyTools.com/#ImportTiddlersPlugin|
|Documentation|http://www.TiddlyTools.com/#ImportTiddlersPluginInfo|
|Version|4.6.2|
|Author|Eric Shulman|
|License|http://www.TiddlyTools.com/#LegalStatements|
|~CoreVersion|2.1|
|Type|plugin|
|Description|interactive controls for import/export with filtering.|
Combine tiddlers from any two TiddlyWiki documents. Interactively select and copy tiddlers from another TiddlyWiki source document. Includes prompting for skip, rename, merge or replace actions when importing tiddlers that match existing titles. When done, a list of all imported tiddlers is written into [[ImportedTiddlers]].
!!!!!Documentation
<<<
see [[ImportTiddlersPluginInfo]] for details
<<<
!!!!!interactive control panel
<<<
<<importTiddlers inline>>
{{clear{
^^(see also: [[ImportTiddlers]] shadow tiddler)^^}}}
<<<
!!!!!Revisions
<<<
2011.02.14 4.6.2 fix OSX error: use picker.file.path
2009.10.10 4.6.1 in createImportPanel, Use {{{window.Components}}} instead of {{{config.browser.isGecko}}} to avoid applying FF3 'file browse' fixup in Chrome.
2009.10.06 4.6.0 added createTiddlerFromFile (import text files)
|please see [[ImportTiddlersPluginInfo]] for additional revision details|
2005.07.20 1.0.0 Initial Release
<<<
!!!!!Code
***/
//{{{
version.extensions.ImportTiddlersPlugin= {major: 4, minor: 6, revision: 2, date: new Date(2011,2,14)};
// IE needs explicit global scoping for functions/vars called from browser events
window.onClickImportButton=onClickImportButton;
window.refreshImportList=refreshImportList;
// default cookie/option values
if (!config.options.chkImportReport) config.options.chkImportReport=true;
// default shadow definition
config.shadowTiddlers.ImportTiddlers='<<importTiddlers inline>>';
// use shadow tiddler content in backstage panel
if (config.tasks) config.tasks.importTask.content='<<tiddler ImportTiddlers>>' // TW2.2 or above
//}}}
//{{{
// backward-compatiblity for TW2.0.x and TW1.2.x
if (config.macros.importTiddlers==undefined) config.macros.importTiddlers={};
if (typeof merge=='undefined') {
function merge(dst,src,preserveExisting) {
for(var i in src) { if(!preserveExisting || dst[i] === undefined) dst[i] = src[i]; }
return dst;
}
}
if (config.browser.isGecko===undefined)
config.browser.isGecko=(config.userAgent.indexOf('gecko')!=-1);
//}}}
//{{{
merge(config.macros.importTiddlers,{
$: function(id) { return document.getElementById(id); }, // abbreviation
label: 'import tiddlers',
prompt: 'Copy tiddlers from another document',
openMsg: 'Opening %0',
openErrMsg: 'Could not open %0 - error=%1',
readMsg: 'Read %0 bytes from %1',
convertUTF8: 'This TW version is %0 (<2.52) => call convertUTF8ToUnicode',
noConvertUTF8: 'This TW version is %0 (>=2.52) => skip convertUTF8ToUnicode()',
foundMsg: 'Found %0 tiddlers in %1',
filterMsg: "Filtered %0 tiddlers matching '%1'",
summaryMsg: '%0 tiddler%1 in the list',
summaryFilteredMsg: '%0 of %1 tiddler%2 in the list',
plural: 's are',
single: ' is',
countMsg: '%0 tiddlers selected for import',
processedMsg: 'Processed %0 tiddlers',
importedMsg: 'Imported %0 of %1 tiddlers from %2',
loadText: 'please load a document...',
closeText: 'close',
doneText: 'done',
startText: 'import',
stopText: 'stop',
local: true, // default to import from local file
src: '', // path/filename or URL of document to import (retrieved from SiteUrl)
proxy: '', // URL for remote proxy script (retrieved from SiteProxy)
useProxy: false, // use specific proxy script in front of remote URL
inbound: null, // hash-indexed array of tiddlers from other document
newTags: '', // text of tags added to imported tiddlers
addTags: true, // add new tags to imported tiddlers
listsize: 10, // # of lines to show in imported tiddler list
importTags: true, // include tags from remote source document when importing a tiddler
keepTags: true, // retain existing tags when replacing a tiddler
sync: false, // add 'server' fields to imported tiddlers (for sync function)
lastFilter: '', // most recent filter (URL hash) applied
lastAction: null, // most recent collision button performed
index: 0, // current processing index in import list
sort: '' // sort order for imported tiddler listbox
});
//}}}
//{{{
// hijack core macro handler
if (config.macros.importTiddlers.coreHandler==undefined)
config.macros.importTiddlers.coreHandler=config.macros.importTiddlers.handler;
config.macros.importTiddlers.handler = function(place,macroName,params,wikifier,paramString,tiddler) {
if (!params[0] || params[0].toLowerCase()=='core') { // default to built in
if (config.macros.importTiddlers.coreHandler)
config.macros.importTiddlers.coreHandler.apply(this,arguments);
else
createTiddlyButton(place,this.label,this.prompt,onClickImportMenu);
} else if (params[0]=='link') { // show link to floating panel
createTiddlyButton(place,params[1]||this.label,params[2]||this.prompt,onClickImportMenu);
} else if (params[0]=='inline') {// show panel as INLINE tiddler content
createImportPanel(place);
this.$('importPanel').style.position='static';
this.$('importPanel').style.display='block';
} else if (config.macros.loadTiddlers)
config.macros.loadTiddlers.handler(place,macroName,params); // any other params: loadtiddlers
}
//}}}
//{{{
// Handle link click to create/show/hide control panel
function onClickImportMenu(e) { var e=e||window.event;
var parent=resolveTarget(e).parentNode;
var panel=document.getElementById('importPanel');
if (panel==undefined || panel.parentNode!=parent) panel=createImportPanel(parent);
var isOpen=panel.style.display=='block';
if(config.options.chkAnimate)
anim.startAnimating(new Slider(panel,!isOpen,false,'none'));
else
panel.style.display=isOpen?'none':'block';
e.cancelBubble = true; if (e.stopPropagation) e.stopPropagation(); return(false);
}
//}}}
//{{{
// Create control panel: HTML, CSS
function createImportPanel(place) {
var cmi=config.macros.importTiddlers; // abbrev
var panel=cmi.$('importPanel');
if (panel) { panel.parentNode.removeChild(panel); }
setStylesheet(store.getTiddlerText('ImportTiddlersPlugin##css'),'importTiddlers');
panel=createTiddlyElement(place,'span','importPanel',null,null)
panel.innerHTML=store.getTiddlerText('ImportTiddlersPlugin##html');
refreshImportList();
if (!cmi.src.length) cmi.src=store.getTiddlerText('SiteUrl')||'';
cmi.$('importSourceURL').value=cmi.src;
if (!cmi.proxy.length) cmi.proxy=store.getTiddlerText('SiteProxy')||'SiteProxy';
cmi.$('importSiteProxy').value=cmi.proxy;
if (window.Components) { // FF3 FIXUP
cmi.$('fileImportSource').style.display='none';
cmi.$('importLocalPanelFix').style.display='block';
}
cmi.$('chkSync').checked=cmi.sync;
cmi.$('chkImportTags').checked=cmi.importTags;
cmi.$('chkKeepTags').checked=cmi.keepTags;
cmi.$('chkAddTags').checked=cmi.addTags;
cmi.$('txtNewTags').value=cmi.newTags;
cmi.$('txtNewTags').style.display=cmi.addTags?'block':'none';
cmi.$('chkSync').checked=cmi.sync;
cmi.$('chkImportReport').checked=config.options.chkImportReport;
return panel;
}
//}}}
//{{{
// process control interactions
function onClickImportButton(which,event) {
var cmi=config.macros.importTiddlers; // abbreviation
var list=cmi.$('importList'); if (!list) return false;
var thePanel=cmi.$('importPanel');
var theCollisionPanel=cmi.$('importCollisionPanel');
var theNewTitle=cmi.$('importNewTitle');
var count=0;
switch (which.id)
{
case 'importFromFile': // show local panel
case 'importFromWeb': // show HTTP panel
cmi.local=(which.id=='importFromFile');
cmi.showPanel('importLocalPanel',cmi.local);
cmi.showPanel('importHTTPPanel',!cmi.local);
break;
case 'importOptions': // show/hide options panel
cmi.showPanel('importOptionsPanel',cmi.$('importOptionsPanel').style.display=='none');
break;
case 'fileImportSource':
case 'importLoad': // load import source into hidden frame
importReport(); // if an import was in progress, generate a report
cmi.inbound=null; // clear the imported tiddler buffer
refreshImportList(); // reset/resize the listbox
if (cmi.src=='') break;
// Load document, read it's DOM and fill the list
cmi.loadRemoteFile(cmi.src,cmi.filterTiddlerList);
break;
case 'importSelectFeed': // select a pre-defined systemServer feed URL
var p=Popup.create(which); if (!p) return false;
var tids=store.getTaggedTiddlers('systemServer');
if (!tids.length)
createTiddlyText(createTiddlyElement(p,'li'),'no pre-defined server feeds');
for (var t=0; t<tids.length; t++) {
var u=store.getTiddlerSlice(tids[t].title,'URL');
var d=store.getTiddlerSlice(tids[t].title,'Description');
if (!d||!d.length) d=store.getTiddlerSlice(tids[t].title,'description');
if (!d||!d.length) d=u;
createTiddlyButton(createTiddlyElement(p,'li'),tids[t].title,d,
function(){
var u=this.getAttribute('url');
document.getElementById('importSourceURL').value=u;
config.macros.importTiddlers.src=u;
document.getElementById('importLoad').onclick();
},
null,null,null,{url:u});
}
Popup.show();
event.cancelBubble = true;
if (event.stopPropagation) event.stopPropagation();
return false;
// create popup with feed list
// onselect, insert feed URL into input field.
break;
case 'importSelectAll': // select all tiddler list items (i.e., not headings)
importReport(); // if an import was in progress, generate a report
for (var t=0,count=0; t < list.options.length; t++) {
if (list.options[t].value=='') continue;
list.options[t].selected=true;
count++;
}
clearMessage(); displayMessage(cmi.countMsg.format([count]));
cmi.$('importStart').disabled=!count;
break;
case 'importSelectNew': // select tiddlers not in current document
importReport(); // if an import was in progress, generate a report
for (var t=0,count=0; t < list.options.length; t++) {
list.options[t].selected=false;
if (list.options[t].value=='') continue;
list.options[t].selected=!store.tiddlerExists(list.options[t].value);
count+=list.options[t].selected?1:0;
}
clearMessage(); displayMessage(cmi.countMsg.format([count]));
cmi.$('importStart').disabled=!count;
break;
case 'importSelectChanges': // select tiddlers that are updated from existing tiddlers
importReport(); // if an import was in progress, generate a report
for (var t=0,count=0; t < list.options.length; t++) {
list.options[t].selected=false;
if (list.options[t].value==''||!store.tiddlerExists(list.options[t].value)) continue;
for (var i=0; i<cmi.inbound.length; i++) // find matching inbound tiddler
{ var inbound=cmi.inbound[i]; if (inbound.title==list.options[t].value) break; }
list.options[t].selected=(inbound.modified-store.getTiddler(list.options[t].value).modified>0); // updated tiddler
count+=list.options[t].selected?1:0;
}
clearMessage(); displayMessage(cmi.countMsg.format([count]));
cmi.$('importStart').disabled=!count;
break;
case 'importSelectDifferences': // select tiddlers that are new or different from existing tiddlers
importReport(); // if an import was in progress, generate a report
for (var t=0,count=0; t < list.options.length; t++) {
list.options[t].selected=false;
if (list.options[t].value=='') continue;
if (!store.tiddlerExists(list.options[t].value)) { list.options[t].selected=true; count++; continue; }
for (var i=0; i<cmi.inbound.length; i++) // find matching inbound tiddler
{ var inbound=cmi.inbound[i]; if (inbound.title==list.options[t].value) break; }
list.options[t].selected=(inbound.modified-store.getTiddler(list.options[t].value).modified!=0); // changed tiddler
count+=list.options[t].selected?1:0;
}
clearMessage(); displayMessage(cmi.countMsg.format([count]));
cmi.$('importStart').disabled=!count;
break;
case 'importApplyFilter': // filter list to include only matching tiddlers
importReport(); // if an import was in progress, generate a report
clearMessage();
if (!cmi.all) // no tiddlers loaded = '0 selected'
{ displayMessage(cmi.countMsg.format([0])); return false; }
var hash=cmi.$('importLastFilter').value;
cmi.inbound=cmi.filterByHash('#'+hash,cmi.all);
refreshImportList(); // reset/resize the listbox
break;
case 'importStart': // initiate the import processing
importReport(); // if an import was in progress, generate a report
cmi.$('importApplyToAll').checked=false;
cmi.$('importStart').value=cmi.stopText;
if (cmi.index>0) cmi.index=-1; // stop processing
else cmi.index=importTiddlers(0); // or begin processing
importStopped();
break;
case 'importClose': // unload imported tiddlers or hide the import control panel
// if imported tiddlers not loaded, close the import control panel
if (!cmi.inbound) { thePanel.style.display='none'; break; }
importReport(); // if an import was in progress, generate a report
cmi.inbound=null; // clear the imported tiddler buffer
refreshImportList(); // reset/resize the listbox
break;
case 'importSkip': // don't import the tiddler
cmi.lastAction=which;
var theItem = list.options[cmi.index];
for (var j=0;j<cmi.inbound.length;j++)
if (cmi.inbound[j].title==theItem.value) break;
var theImported = cmi.inbound[j];
theImported.status='skipped after asking'; // mark item as skipped
theCollisionPanel.style.display='none';
cmi.index=importTiddlers(cmi.index+1); // resume with NEXT item
importStopped();
break;
case 'importRename': // change name of imported tiddler
cmi.lastAction=which;
var theItem = list.options[cmi.index];
for (var j=0;j<cmi.inbound.length;j++)
if (cmi.inbound[j].title==theItem.value) break;
var theImported = cmi.inbound[j];
theImported.status = 'renamed from '+theImported.title; // mark item as renamed
theImported.set(theNewTitle.value,null,null,null,null); // change the tiddler title
theItem.value = theNewTitle.value; // change the listbox item text
theItem.text = theNewTitle.value; // change the listbox item text
theCollisionPanel.style.display='none';
cmi.index=importTiddlers(cmi.index); // resume with THIS item
importStopped();
break;
case 'importMerge': // join existing and imported tiddler content
cmi.lastAction=which;
var theItem = list.options[cmi.index];
for (var j=0;j<cmi.inbound.length;j++)
if (cmi.inbound[j].title==theItem.value) break;
var theImported = cmi.inbound[j];
var theExisting = store.getTiddler(theItem.value);
var theText = theExisting.text+'\n----\n^^merged from: ';
theText +='[['+cmi.src+'#'+theItem.value+'|'+cmi.src+'#'+theItem.value+']]^^\n';
theText +='^^'+theImported.modified.toLocaleString()+' by '+theImported.modifier+'^^\n'+theImported.text;
var theDate = new Date();
var theTags = theExisting.getTags()+' '+theImported.getTags();
theImported.set(null,theText,null,theDate,theTags);
theImported.status = 'merged with '+theExisting.title; // mark item as merged
theImported.status += ' - '+theExisting.modified.formatString('MM/DD/YYYY 0hh:0mm:0ss');
theImported.status += ' by '+theExisting.modifier;
theCollisionPanel.style.display='none';
cmi.index=importTiddlers(cmi.index); // resume with this item
importStopped();
break;
case 'importReplace': // substitute imported tiddler for existing tiddler
cmi.lastAction=which;
var theItem = list.options[cmi.index];
for (var j=0;j<cmi.inbound.length;j++)
if (cmi.inbound[j].title==theItem.value) break;
var theImported = cmi.inbound[j];
var theExisting = store.getTiddler(theItem.value);
theImported.status = 'replaces '+theExisting.title; // mark item for replace
theImported.status += ' - '+theExisting.modified.formatString('MM/DD/YYYY 0hh:0mm:0ss');
theImported.status += ' by '+theExisting.modifier;
theCollisionPanel.style.display='none';
cmi.index=importTiddlers(cmi.index); // resume with THIS item
importStopped();
break;
case 'importListSmaller': // decrease current listbox size, minimum=5
if (list.options.length==1) break;
list.size-=(list.size>5)?1:0;
cmi.listsize=list.size;
break;
case 'importListLarger': // increase current listbox size, maximum=number of items in list
if (list.options.length==1) break;
list.size+=(list.size<list.options.length)?1:0;
cmi.listsize=list.size;
break;
case 'importListMaximize': // toggle listbox size between current and maximum
if (list.options.length==1) break;
list.size=(list.size==list.options.length)?cmi.listsize:list.options.length;
break;
}
}
//}}}
//{{{
config.macros.importTiddlers.showPanel=function(place,show,skipAnim) {
if (typeof place=='string') var place=document.getElementById(place);
if (!place||!place.style) return;
if(!skipAnim && anim && config.options.chkAnimate) anim.startAnimating(new Slider(place,show,false,'none'));
else place.style.display=show?'block':'none';
}
//}}}
//{{{
function refreshImportList(selectedIndex) {
var cmi=config.macros.importTiddlers; // abbrev
var list=cmi.$('importList'); if (!list) return;
// if nothing to show, reset list content and size
if (!cmi.inbound) {
while (list.length > 0) { list.options[0] = null; }
list.options[0]=new Option(cmi.loadText,'',false,false);
list.size=cmi.listsize;
cmi.$('importLoad').disabled=false;
cmi.$('importLoad').style.display='inline';
cmi.$('importStart').disabled=true;
cmi.$('importOptions').disabled=true;
cmi.$('importOptions').style.display='none';
cmi.$('fileImportSource').disabled=false;
cmi.$('importFromFile').disabled=false;
cmi.$('importFromWeb').disabled=false;
cmi.$('importStart').value=cmi.startText;
cmi.$('importClose').value=cmi.doneText;
cmi.$('importSelectPanel').style.display='none';
cmi.$('importOptionsPanel').style.display='none';
return;
}
// there are inbound tiddlers loaded...
cmi.$('importLoad').disabled=true;
cmi.$('importLoad').style.display='none';
cmi.$('importOptions').style.display='inline';
cmi.$('importOptions').disabled=false;
cmi.$('fileImportSource').disabled=true;
cmi.$('importFromFile').disabled=true;
cmi.$('importFromWeb').disabled=true;
cmi.$('importClose').value=cmi.closeText;
if (cmi.$('importSelectPanel').style.display=='none')
cmi.showPanel('importSelectPanel',true);
// get the sort order
if (!selectedIndex) selectedIndex=0;
if (selectedIndex==0) cmi.sort='title'; // heading
if (selectedIndex==1) cmi.sort='title';
if (selectedIndex==2) cmi.sort='modified';
if (selectedIndex==3) cmi.sort='tags';
if (selectedIndex>3) {
// display selected tiddler count
for (var t=0,count=0; t < list.options.length; t++) {
if (!list.options[t].selected) continue;
if (list.options[t].value!='')
count+=1;
else { // if heading is selected, deselect it, and then select and count all in section
list.options[t].selected=false;
for ( t++; t<list.options.length && list.options[t].value!=''; t++) {
list.options[t].selected=true;
count++;
}
}
}
clearMessage(); displayMessage(cmi.countMsg.format([count]));
}
cmi.$('importStart').disabled=!count;
if (selectedIndex>3) return; // no refresh needed
// get the alphasorted list of tiddlers
var tiddlers=cmi.inbound;
tiddlers.sort(function (a,b) {if(a['title'] == b['title']) return(0); else return (a['title'] < b['title']) ? -1 : +1; });
// clear current list contents
while (list.length > 0) { list.options[0] = null; }
// add heading and control items to list
var i=0;
var indent=String.fromCharCode(160)+String.fromCharCode(160);
if (cmi.all.length==tiddlers.length)
var summary=cmi.summaryMsg.format([tiddlers.length,(tiddlers.length!=1)?cmi.plural:cmi.single]);
else
var summary=cmi.summaryFilteredMsg.format([tiddlers.length,cmi.all.length,(cmi.all.length!=1)?cmi.plural:cmi.single]);
list.options[i++]=new Option(summary,'',false,false);
list.options[i++]=new Option(((cmi.sort=='title' )?'>':indent)+' [by title]','',false,false);
list.options[i++]=new Option(((cmi.sort=='modified')?'>':indent)+' [by date]','',false,false);
list.options[i++]=new Option(((cmi.sort=='tags')?'>':indent)+' [by tags]','',false,false);
// output the tiddler list
switch(cmi.sort) {
case 'title':
for(var t = 0; t < tiddlers.length; t++)
list.options[i++] = new Option(tiddlers[t].title,tiddlers[t].title,false,false);
break;
case 'modified':
// sort descending for newest date first
tiddlers.sort(function (a,b) {if(a['modified'] == b['modified']) return(0); else return (a['modified'] > b['modified']) ? -1 : +1; });
var lastSection = '';
for(var t = 0; t < tiddlers.length; t++) {
var tiddler = tiddlers[t];
var theSection = tiddler.modified.toLocaleDateString();
if (theSection != lastSection) {
list.options[i++] = new Option(theSection,'',false,false);
lastSection = theSection;
}
list.options[i++] = new Option(indent+indent+tiddler.title,tiddler.title,false,false);
}
break;
case 'tags':
var theTitles = {}; // all tiddler titles, hash indexed by tag value
var theTags = new Array();
for(var t=0; t<tiddlers.length; t++) {
var title=tiddlers[t].title;
var tags=tiddlers[t].tags;
if (!tags || !tags.length) {
if (theTitles['untagged']==undefined) { theTags.push('untagged'); theTitles['untagged']=new Array(); }
theTitles['untagged'].push(title);
}
else for(var s=0; s<tags.length; s++) {
if (theTitles[tags[s]]==undefined) { theTags.push(tags[s]); theTitles[tags[s]]=new Array(); }
theTitles[tags[s]].push(title);
}
}
theTags.sort();
for(var tagindex=0; tagindex<theTags.length; tagindex++) {
var theTag=theTags[tagindex];
list.options[i++]=new Option(theTag,'',false,false);
for(var t=0; t<theTitles[theTag].length; t++)
list.options[i++]=new Option(indent+indent+theTitles[theTag][t],theTitles[theTag][t],false,false);
}
break;
}
list.selectedIndex=selectedIndex; // select current control item
if (list.size<cmi.listsize) list.size=cmi.listsize;
if (list.size>list.options.length) list.size=list.options.length;
}
//}}}
//{{{
// re-entrant processing for handling import with interactive collision prompting
function importTiddlers(startIndex) {
var cmi=config.macros.importTiddlers; // abbrev
if (!cmi.inbound) return -1;
var list=cmi.$('importList'); if (!list) return;
var t;
// if starting new import, reset import status flags
if (startIndex==0)
for (var t=0;t<cmi.inbound.length;t++)
cmi.inbound[t].status='';
for (var i=startIndex; i<list.options.length; i++) {
// if list item is not selected or is a heading (i.e., has no value), skip it
if ((!list.options[i].selected) || ((t=list.options[i].value)==''))
continue;
for (var j=0;j<cmi.inbound.length;j++)
if (cmi.inbound[j].title==t) break;
var inbound = cmi.inbound[j];
var theExisting = store.getTiddler(inbound.title);
// avoid redundant import for tiddlers that are listed multiple times (when 'by tags')
if (inbound.status=='added')
continue;
// don't import the 'ImportedTiddlers' history from the other document...
if (inbound.title=='ImportedTiddlers')
continue;
// if tiddler exists and import not marked for replace or merge, stop importing
if (theExisting && (inbound.status.substr(0,7)!='replace') && (inbound.status.substr(0,5)!='merge'))
return i;
// assemble tags (remote + existing + added)
var newTags = '';
if (cmi.importTags)
newTags+=inbound.getTags() // import remote tags
if (cmi.keepTags && theExisting)
newTags+=' '+theExisting.getTags(); // keep existing tags
if (cmi.addTags && cmi.newTags.trim().length)
newTags+=' '+cmi.newTags; // add new tags
inbound.set(null,null,null,null,newTags.trim());
// set the status to 'added' (if not already set by the 'ask the user' UI)
inbound.status=(inbound.status=='')?'added':inbound.status;
// set sync fields
if (cmi.sync) {
if (!inbound.fields) inbound.fields={}; // for TW2.1.x backward-compatibility
inbound.fields['server.page.revision']=inbound.modified.convertToYYYYMMDDHHMM();
inbound.fields['server.type']='file';
inbound.fields['server.host']=(cmi.local&&!cmi.src.startsWith('file:')?'file:///':'')+cmi.src;
}
// do the import!
store.suspendNotifications();
store.saveTiddler(inbound.title, inbound.title, inbound.text, inbound.modifier, inbound.modified, inbound.tags, inbound.fields, true, inbound.created);
store.fetchTiddler(inbound.title).created = inbound.created; // force creation date to imported value (needed for TW2.1.x and earlier)
store.resumeNotifications();
}
return(-1); // signals that we really finished the entire list
}
function importStopped() {
var cmi=config.macros.importTiddlers; // abbrev
var list=cmi.$('importList'); if (!list) return;
var theNewTitle=cmi.$('importNewTitle');
if (cmi.index==-1){
cmi.$('importStart').value=cmi.startText;
importReport(); // import finished... generate the report
} else {
// import collision...
// show the collision panel and set the title edit field
cmi.$('importStart').value=cmi.stopText;
cmi.showPanel('importCollisionPanel',true);
theNewTitle.value=list.options[cmi.index].value;
if (cmi.$('importApplyToAll').checked && cmi.lastAction && cmi.lastAction.id!='importRename')
onClickImportButton(cmi.lastAction);
}
}
//}}}
//{{{
function importReport() {
var cmi=config.macros.importTiddlers; // abbrev
if (!cmi.inbound) return;
// if import was not completed, the collision panel will still be open... close it now.
var panel=cmi.$('importCollisionPanel'); if (panel) panel.style.display='none';
// get the alphasorted list of tiddlers
var tiddlers = cmi.inbound;
// gather the statistics
var count=0; var total=0;
for (var t=0; t<tiddlers.length; t++) {
if (!tiddlers[t].status || !tiddlers[t].status.trim().length) continue;
if (tiddlers[t].status.substr(0,7)!='skipped') count++;
total++;
}
// generate a report
if (total) displayMessage(cmi.processedMsg.format([total]));
if (count && config.options.chkImportReport) {
// get/create the report tiddler
var theReport = store.getTiddler('ImportedTiddlers');
if (!theReport) { theReport=new Tiddler(); theReport.title='ImportedTiddlers'; theReport.text=''; }
// format the report content
var now = new Date();
var newText = 'On '+now.toLocaleString()+', '+config.options.txtUserName
newText +=' imported '+count+' tiddler'+(count==1?'':'s')+' from\n[['+cmi.src+'|'+cmi.src+']]:\n';
if (cmi.addTags && cmi.newTags.trim().length)
newText += 'imported tiddlers were tagged with: "'+cmi.newTags+'"\n';
newText += '<<<\n';
for (var t=0; t<tiddlers.length; t++) if (tiddlers[t].status)
newText += '#[['+tiddlers[t].title+']] - '+tiddlers[t].status+'\n';
newText += '<<<\n';
// update the ImportedTiddlers content and show the tiddler
theReport.text = newText+((theReport.text!='')?'\n----\n':'')+theReport.text;
theReport.modifier = config.options.txtUserName;
theReport.modified = new Date();
store.saveTiddler(theReport.title, theReport.title, theReport.text, theReport.modifier, theReport.modified, theReport.tags, theReport.fields);
story.displayTiddler(null,theReport.title,1,null,null,false);
story.refreshTiddler(theReport.title,1,true);
}
// reset status flags
for (var t=0; t<cmi.inbound.length; t++) cmi.inbound[t].status='';
// mark document as dirty and let display update as needed
if (count) { store.setDirty(true); store.notifyAll(); }
// always show final message when tiddlers were actually loaded
if (count) displayMessage(cmi.importedMsg.format([count,tiddlers.length,cmi.src.replace(/%20/g,' ')]));
}
//}}}
//{{{
// // File and XMLHttpRequest I/O
config.macros.importTiddlers.askForFilename=function(here) {
var msg=here.title; // use tooltip as dialog box message
var path=getLocalPath(document.location.href);
var slashpos=path.lastIndexOf('/'); if (slashpos==-1) slashpos=path.lastIndexOf('\\');
if (slashpos!=-1) path = path.substr(0,slashpos+1); // remove filename from path, leave the trailing slash
var file='';
var result='';
if(window.Components) { // moz
try {
netscape.security.PrivilegeManager.enablePrivilege('UniversalXPConnect');
var nsIFilePicker = window.Components.interfaces.nsIFilePicker;
var picker = Components.classes['@mozilla.org/filepicker;1'].createInstance(nsIFilePicker);
picker.init(window, msg, nsIFilePicker.modeOpen);
var thispath = Components.classes['@mozilla.org/file/local;1'].createInstance(Components.interfaces.nsILocalFile);
thispath.initWithPath(path);
picker.displayDirectory=thispath;
picker.defaultExtension='html';
picker.defaultString=file;
picker.appendFilters(nsIFilePicker.filterAll|nsIFilePicker.filterText|nsIFilePicker.filterHTML);
if (picker.show()!=nsIFilePicker.returnCancel) var result=picker.file.path;
}
catch(e) { alert('error during local file access: '+e.toString()) }
}
else { // IE
try { // XPSP2 IE only
var s = new ActiveXObject('UserAccounts.CommonDialog');
s.Filter='All files|*.*|Text files|*.txt|HTML files|*.htm;*.html|';
s.FilterIndex=3; // default to HTML files;
s.InitialDir=path;
s.FileName=file;
if (s.showOpen()) var result=s.FileName;
}
catch(e) { // fallback
var result=prompt(msg,path+file);
}
}
return result;
}
config.macros.importTiddlers.loadRemoteFile = function(src,callback) {
if (src==undefined || !src.length) return null; // filename is required
var original=src; // URL as specified
var hashpos=src.indexOf('#'); if (hashpos!=-1) src=src.substr(0,hashpos); // URL with #... suffix removed (needed for IE)
clearMessage();
displayMessage(this.openMsg.format([src.replace(/%20/g,' ')]));
if (src.substr(0,5)!='http:' && src.substr(0,5)!='file:') { // if not a URL, read from local filesystem
var txt=loadFile(src);
if (!txt) { // file didn't load, might be relative path.. try fixup
var pathPrefix=document.location.href; // get current document path and trim off filename
var slashpos=pathPrefix.lastIndexOf('/'); if (slashpos==-1) slashpos=pathPrefix.lastIndexOf('\\');
if (slashpos!=-1 && slashpos!=pathPrefix.length-1) pathPrefix=pathPrefix.substr(0,slashpos+1);
src=pathPrefix+src;
if (pathPrefix.substr(0,5)!='http:') src=getLocalPath(src);
var txt=loadFile(src);
}
if (!txt) { // file still didn't load, report error
displayMessage(config.macros.importTiddlers.openErrMsg.format([src.replace(/%20/g,' '),'(filesystem error)']));
} else {
displayMessage(config.macros.importTiddlers.readMsg.format([txt.length,src.replace(/%20/g,' ')]));
var vers = version.major+version.minor*.1+version.revision*.01;
if (vers < 2.52) {
displayMessage(config.macros.importTiddlers.convertUTF8.format([vers]));
txt=convertUTF8ToUnicode(txt);
} else {
displayMessage(config.macros.importTiddlers.noConvertUTF8.format([vers]));
}
if (callback) callback(true,original,txt,src,null);
}
} else {
doHttp('GET',src,null,null,config.options.txtRemoteUsername,config.options.txtRemotePassword,callback,original,null);
}
}
config.macros.importTiddlers.readTiddlersFromHTML=function(html){
var remoteStore=new TiddlyWiki();
remoteStore.importTiddlyWiki(html);
return remoteStore.getTiddlers('title');
}
config.macros.importTiddlers.readTiddlersFromCSV=function(CSV){
var remoteStore=new TiddlyWiki();
// GET NAMES
var lines=CSV.replace(/\r/g,'').split('\n');
var names=lines.shift().replace(/"/g,'').split(',');
CSV=lines.join('\n');
// ENCODE commas and newlines within quoted values
var comma='!~comma~!'; var commaRE=new RegExp(comma,'g');
var newline='!~newline~!'; var newlineRE=new RegExp(newline,'g');
CSV=CSV.replace(/"([^"]*?)"/g,
function(x){ return x.replace(/\,/g,comma).replace(/\n/g,newline); });
// PARSE lines
var lines=CSV.split('\n');
for (var i=0; i<lines.length; i++) { if (!lines[i].length) continue;
var values=lines[i].split(',');
// DECODE commas, newlines, and doubled-quotes, and remove enclosing quotes (if any)
for (var v=0; v<values.length; v++)
values[v]=values[v].replace(commaRE,',').replace(newlineRE,'\n')
.replace(/^"|"$/g,'').replace(/""/g,'"');
// EXTRACT tiddler values
var title=''; var text=''; var tags=[]; var fields={};
var created=null; var when=new Date(); var who=config.options.txtUserName;
for (var v=0; v<values.length; v++) { var val=values[v];
if (names[v]) switch(names[v].toLowerCase()) {
case 'title': title=val.replace(/\[\]\|/g,'_'); break;
case 'created': created=new Date(val); break;
case 'modified':when=new Date(val); break;
case 'modifier':who=val; break;
case 'text': text=val; break;
case 'tags': tags=val.readBracketedList(); break;
default: fields[names[v].toLowerCase()]=val; break;
}
}
// CREATE tiddler in temporary store
if (title.length)
remoteStore.saveTiddler(title,title,text,who,when,tags,fields,true,created||when);
}
return remoteStore.getTiddlers('title');
}
config.macros.importTiddlers.createTiddlerFromFile=function(src,txt) {
var t=new Tiddler();
var pos=src.lastIndexOf("/"); if (pos==-1) pos=src.lastIndexOf("\\");
t.title=pos==-1?src:src.substr(pos+1);
t.text=txt;
t.created=t.modified=new Date();
t.modifier=config.options.txtUserName;
if (src.substr(src.length-3,3)=='.js') t.tags=['systemConfig'];
return [t];
}
config.macros.importTiddlers.filterTiddlerList=function(success,params,txt,src,xhr){
var cmi=config.macros.importTiddlers; // abbreviation
var src=src.replace(/%20/g,' ');
if (!success) { displayMessage(cmi.openErrMsg.format([src,xhr.status])); return; }
cmi.all=cmi.readTiddlersFromHTML(txt);
if (!cmi.all||!cmi.all.length) cmi.all=cmi.readTiddlersFromCSV(txt)
if (!cmi.all||!cmi.all.length) cmi.all=cmi.createTiddlerFromFile(src,txt)
var count=cmi.all?cmi.all.length:0;
var querypos=src.lastIndexOf('?'); if (querypos!=-1) src=src.substr(0,querypos);
displayMessage(cmi.foundMsg.format([count,src]));
cmi.inbound=cmi.filterByHash(params,cmi.all); // use full URL including hash (if any)
cmi.$('importLastFilter').value=cmi.lastFilter;
window.refreshImportList(0);
}
config.macros.importTiddlers.filterByHash=function(src,tiddlers){
var hashpos=src.lastIndexOf('#'); if (hashpos==-1) return tiddlers;
var hash=src.substr(hashpos+1); if (!hash.length) return tiddlers;
var tids=[];
var params=hash.parseParams('anon',null,true,false,false);
for (var p=1; p<params.length; p++) {
switch (params[p].name) {
case 'anon':
case 'open':
tids.pushUnique(params[p].value);
break;
case 'tag':
if (store.getMatchingTiddlers) { // for boolean expressions - see MatchTagsPlugin
var r=store.getMatchingTiddlers(params[p].value,null,tiddlers);
for (var t=0; t<r.length; t++) tids.pushUnique(r[t].title);
} else for (var t=0; t<tiddlers.length; t++)
if (tiddlers[t].isTagged(params[p].value))
tids.pushUnique(tiddlers[t].title);
break;
case 'story':
for (var t=0; t<tiddlers.length; t++)
if (tiddlers[t].title==params[p].value) {
tiddlers[t].changed();
for (var s=0; s<tiddlers[t].links.length; s++)
tids.pushUnique(tiddlers[t].links[s]);
break;
}
break;
case 'search':
for (var t=0; t<tiddlers.length; t++)
if (tiddlers[t].text.indexOf(params[p].value)!=-1)
tids.pushUnique(tiddlers[t].title);
break;
}
}
var matches=[];
for (var t=0; t<tiddlers.length; t++)
if (tids.contains(tiddlers[t].title))
matches.push(tiddlers[t]);
displayMessage(config.macros.importTiddlers.filterMsg.format([matches.length,hash]));
config.macros.importTiddlers.lastFilter=hash;
return matches;
}
//}}}
/***
!!!Control panel CSS
//{{{
!css
#importPanel {
display: none; position:absolute; z-index:11; width:35em; right:105%; top:3em;
background-color: #eee; color:#000; font-size: 8pt; line-height:110%;
border:1px solid black; border-bottom-width: 3px; border-right-width: 3px;
padding: 0.5em; margin:0em; -moz-border-radius:1em;-webkit-border-radius:1em;
}
#importPanel a, #importPanel td a { color:#009; display:inline; margin:0px; padding:1px; }
#importPanel table { width:100%; border:0px; padding:0px; margin:0px; font-size:8pt; line-height:110%; background:transparent; }
#importPanel tr { border:0px;padding:0px;margin:0px; background:transparent; }
#importPanel td { color:#000; border:0px;padding:0px;margin:0px; background:transparent; }
#importPanel select { width:100%;margin:0px;font-size:8pt;line-height:110%;}
#importPanel input { width:98%;padding:0px;margin:0px;font-size:8pt;line-height:110%}
#importPanel .box { border:1px solid #000; background-color:#eee; padding:3px 5px; margin-bottom:5px; -moz-border-radius:5px;-webkit-border-radius:5px;}
#importPanel .topline { border-top:1px solid #999; padding-top:2px; margin-top:2px; }
#importPanel .rad { width:auto; }
#importPanel .chk { width:auto; margin:1px;border:0; }
#importPanel .btn { width:auto; }
#importPanel .btn1 { width:98%; }
#importPanel .btn2 { width:48%; }
#importPanel .btn3 { width:32%; }
#importPanel .btn4 { width:23%; }
#importPanel .btn5 { width:19%; }
#importPanel .importButton { padding: 0em; margin: 0px; font-size:8pt; }
#importPanel .importListButton { padding:0em 0.25em 0em 0.25em; color: #000000; display:inline }
#backstagePanel #importPanel { left:10%; right:auto; }
!end
//}}}
!!!Control panel HTML
//{{{
!html
<!-- source and report -->
<table><tr><td align=left>
import from
<input type="radio" class="rad" name="importFrom" id="importFromFile" value="file" CHECKED
onclick="onClickImportButton(this,event)" title="show file controls"> local file
<input type="radio" class="rad" name="importFrom" id="importFromWeb" value="http"
onclick="onClickImportButton(this,event)" title="show web controls"> web server
</td><td align=right>
<input type=checkbox class="chk" id="chkImportReport"
onClick="config.options['chkImportReport']=this.checked;"> create report
</td></tr></table>
<div class="box" id="importSourcePanel" style="margin:.5em">
<div id="importLocalPanel" style="display:block;margin-bottom:2px;"><!-- import from local file -->
enter or browse for source path/filename<br>
<input type="file" id="fileImportSource" size=57 style="width:100%"
onKeyUp="config.macros.importTiddlers.src=this.value"
onChange="config.macros.importTiddlers.src=this.value;document.getElementById('importLoad').onclick()">
<div id="importLocalPanelFix" style="display:none"><!-- FF3 FIXUP -->
<input type="text" id="fileImportSourceFix" style="width:90%"
title="Enter a path/file to import"
onKeyUp="config.macros.importTiddlers.src=this.value"
onChange="config.macros.importTiddlers.src=this.value;document.getElementById('importLoad').onclick()">
<input type="button" id="fileImportSourceFixButton" style="width:7%" value="..."
title="Select a path/file to import"
onClick="var r=config.macros.importTiddlers.askForFilename(this); if (!r||!r.length) return;
document.getElementById('fileImportSourceFix').value=r;
config.macros.importTiddlers.src=r;
document.getElementById('importLoad').onclick()">
</div><!--end FF3 FIXUP-->
</div><!--end local-->
<div id="importHTTPPanel" style="display:none;margin-bottom:2px;"><!-- import from http server -->
<table><tr><td align=left>
enter a URL or <a href="javascript:;" id="importSelectFeed"
onclick="return onClickImportButton(this,event)" title="select a pre-defined 'systemServer' URL">
select a server</a><br>
</td><td align=right>
<input type="checkbox" class="chk" id="importUsePassword"
onClick="config.macros.importTiddlers.usePassword=this.checked;
config.macros.importTiddlers.showPanel('importIDPWPanel',this.checked,true);">password
<input type="checkbox" class="chk" id="importUseProxy"
onClick="config.macros.importTiddlers.useProxy=this.checked;
config.macros.importTiddlers.showPanel('importSiteProxy',this.checked,true);">proxy
</td></tr></table>
<input type="text" id="importSiteProxy" style="display:none;margin-bottom:1px" onfocus="this.select()" value="SiteProxy"
onKeyUp="config.macros.importTiddlers.proxy=this.value"
onChange="config.macros.importTiddlers.proxy=this.value;">
<input type="text" id="importSourceURL" onfocus="this.select()" value="SiteUrl"
onKeyUp="config.macros.importTiddlers.src=this.value"
onChange="config.macros.importTiddlers.src=this.value;">
<div id="importIDPWPanel" style="text-align:center;margin-top:2px;display:none";>
username: <input type=text id="txtImportID" style="width:25%"
onChange="config.options.txtRemoteUsername=this.value;">
password: <input type=password id="txtImportPW" style="width:25%"
onChange="config.options.txtRemotePassword=this.value;">
</div><!--end idpw-->
</div><!--end http-->
</div><!--end source-->
<div class="box" id="importSelectPanel" style="display:none;margin:.5em;">
<table><tr><td align=left>
select:
<a href="javascript:;" id="importSelectAll"
onclick="return onClickImportButton(this)" title="SELECT all tiddlers">
all</a>
<a href="javascript:;" id="importSelectNew"
onclick="return onClickImportButton(this)" title="SELECT tiddlers not already in destination document">
added</a>
<a href="javascript:;" id="importSelectChanges"
onclick="return onClickImportButton(this)" title="SELECT tiddlers that have been updated in source document">
changes</a>
<a href="javascript:;" id="importSelectDifferences"
onclick="return onClickImportButton(this)" title="SELECT tiddlers that have been added or are different from existing tiddlers">
differences</a>
</td><td align=right>
<a href="javascript:;" id="importListSmaller"
onclick="return onClickImportButton(this)" title="SHRINK list size">
– </a>
<a href="javascript:;" id="importListLarger"
onclick="return onClickImportButton(this)" title="GROW list size">
+ </a>
<a href="javascript:;" id="importListMaximize"
onclick="return onClickImportButton(this)" title="MAXIMIZE/RESTORE list size">
= </a>
</td></tr></table>
<select id="importList" size=8 multiple
onchange="setTimeout('refreshImportList('+this.selectedIndex+')',1)">
<!-- NOTE: delay refresh so list is updated AFTER onchange event is handled -->
</select>
<div style="text-align:center">
<a href="javascript:;"
title="click for help using filters..."
onclick="alert('A filter consists of one or more space-separated combinations of: tiddlertitle, tag:[[tagvalue]], tag:[[tag expression]] (requires MatchTagsPlugin), story:[[TiddlerName]], and/or search:[[searchtext]]. Use a blank filter to restore the list of all tiddlers.'); return false;"
>filter</a>
<input type="text" id="importLastFilter" style="margin-bottom:1px; width:65%"
title="Enter a combination of one or more filters. Use a blank filter for all tiddlers."
onfocus="this.select()" value=""
onKeyUp="config.macros.importTiddlers.lastFilter=this.value"
onChange="config.macros.importTiddlers.lastFilter=this.value;">
<input type="button" id="importApplyFilter" style="width:20%" value="apply"
title="filter list of tiddlers to include only those that match certain criteria"
onclick="return onClickImportButton(this)">
</div>
</div><!--end select-->
<div class="box" id="importOptionsPanel" style="text-align:center;margin:.5em;display:none;">
apply tags: <input type=checkbox class="chk" id="chkImportTags" checked
onClick="config.macros.importTiddlers.importTags=this.checked;">from source
<input type=checkbox class="chk" id="chkKeepTags" checked
onClick="config.macros.importTiddlers.keepTags=this.checked;">keep existing
<input type=checkbox class="chk" id="chkAddTags"
onClick="config.macros.importTiddlers.addTags=this.checked;
config.macros.importTiddlers.showPanel('txtNewTags',this.checked,false);
if (this.checked) document.getElementById('txtNewTags').focus();">add tags<br>
<input type=text id="txtNewTags" style="margin-top:4px;display:none;" size=15 onfocus="this.select()"
title="enter tags to be added to imported tiddlers"
onKeyUp="config.macros.importTiddlers.newTags=this.value;
document.getElementById('chkAddTags').checked=this.value.length>0;" autocomplete=off>
<nobr><input type=checkbox class="chk" id="chkSync"
onClick="config.macros.importTiddlers.sync=this.checked;">
link tiddlers to source document (for sync later)</nobr>
</div><!--end options-->
<div id="importButtonPanel" style="text-align:center">
<input type=button id="importLoad" class="importButton btn3" value="open"
title="load listbox with tiddlers from source document"
onclick="onClickImportButton(this)">
<input type=button id="importOptions" class="importButton btn3" value="options..."
title="set options for tags, sync, etc."
onclick="onClickImportButton(this)">
<input type=button id="importStart" class="importButton btn3" value="import"
title="start/stop import of selected source tiddlers into current document"
onclick="onClickImportButton(this)">
<input type=button id="importClose" class="importButton btn3" value="done"
title="clear listbox or hide control panel"
onclick="onClickImportButton(this)">
</div>
<div class="none" id="importCollisionPanel" style="display:none;margin:.5em 0 .5em .5em;">
<table><tr><td style="width:65%" align="left">
<table><tr><td align=left>
tiddler already exists:
</td><td align=right>
<input type=checkbox class="chk" id="importApplyToAll"
onclick="document.getElementById('importRename').disabled=this.checked;"
checked>apply to all
</td></tr></table>
<input type=text id="importNewTitle" size=15 autocomplete=off">
</td><td style="width:34%" align="center">
<input type=button id="importMerge"
class="importButton" style="width:47%" value="merge"
title="append the incoming tiddler to the existing tiddler"
onclick="onClickImportButton(this)"><!--
--><input type=button id="importSkip"
class="importButton" style="width:47%" value="skip"
title="do not import this tiddler"
onclick="onClickImportButton(this)"><!--
--><br><input type=button id="importRename"
class="importButton" style="width:47%" value="rename"
title="rename the incoming tiddler"
onclick="onClickImportButton(this)"><!--
--><input type=button id="importReplace"
class="importButton" style="width:47%" value="replace"
title="discard the existing tiddler"
onclick="onClickImportButton(this)">
</td></tr></table>
</div><!--end collision-->
!end
//}}}
***/
On mar. 01 oct. 2013 13:22:32 CEST, french-postaggers imported 57 tiddlers from
[[POS_Taggers.html|POS_Taggers.html]]:
<<<
#[[Alpage Linguistic Processing Chain]] - added
#[[Bonsai]] - added
#[[CC Tagset]] - added
#[[CoNLL data format]] - added
#[[DefaultTiddlers]] - added
#[[EditTemplate]] - added
#[[FieldsEditorPlugin]] - added
#[[Freeling]] - added
#[[French TreeTagger Part-of-Speech Tags]] - added
#[[Hunpos]] - added
#[[InlineJavascriptPluginInfo]] - added
#[[Input, Output and Co.]] - added
#[[JSafran]] - added
#[[LGTagger]] - added
#[[LIA_TAGG]] - added
#[[LanguageTool]] - added
#[[LanguageTool English Tagset]] - added
#[[LanguageTool French Tagset]] - added
#[[ListTagsPlugin]] - added
#[[MElt (ALPAGE Linguistic Workbench)]] - added
#[[Main Page]] - added
#[[MainMenu]] - added
#[[Mallet]] - added
#[[Malt-TAB]] - added
#[[MaltParser]] - added
#[[MatchTagsPlugin]] - added
#[[MatchTagsPluginInfo]] - added
#[[Morfette]] - added
#[[Multext tools / TATOO]] - added
#[[POS tagger links]] - added
#[[Penn Treebank English POS tag set]] - added
#[[SEM]] - added
#[[SEM Tagset]] - added
#[[SVMTool]] - added
#[[SYGFRAN]] - added
#[[SiteSubtitle]] - skipped after asking
#[[SiteTitle]] - skipped after asking
#[[Stanford POS Tagger]] - added
#[[StyleSheet]] - added
#[[TATOO]] - added
#[[TaggerPluginDocumentation]] - added
#[[TaggerPluginSource]] - added
#[[TagsTreePlugin]] - added
#[[TnT]] - added
#[[ToolbarCommands]] - added
#[[TreeTagger]] - added
#[[UIMA]] - added
#[[ViewTemplate]] - added
#[[Wall Street Journal (WSJ) Tagset]] - added
#[[exemple in:sent]] - added
#[[lang:]] - added
#[[lang:...]] - added
#[[lang:AR]] - added
#[[lang:CAT]] - added
#[[lang:EN]] - added
#[[lang:ES]] - added
#[[lang:FR]] - added
#[[languagetool output start]] - added
#[[openNLP]] - added
<<<
/***
|Name|InlineJavascriptPluginInfo|
|Source|http://www.TiddlyTools.com/#InlineJavascriptPlugin|
|Documentation|http://www.TiddlyTools.com/#InlineJavascriptPluginInfo|
|Version|1.9.6|
|Author|Eric Shulman|
|License|http://www.TiddlyTools.com/#LegalStatements|
|~CoreVersion|2.1|
|Type|documentation|
|Description|Documentation for InlineJavascriptPlugin|
''Call directly into TW core utility routines, define new functions, calculate values, add dynamically-generated TiddlyWiki-formatted output'' into tiddler content, or perform any other programmatic actions each time the tiddler is rendered.
!!!!!Usage
<<<
This plugin adds wiki syntax for surrounding tiddler content with {{{<script>}}} and {{{</script>}}} markers, so that it can be recognized as embedded javascript code. When a tiddler is rendered, the plugin automatically invokes any embedded scripts, which can be used to construct and return dynamically-generated output that is inserted into the tiddler content.
{{{
<script type="..." src="..." label="..." title="..." key="..." show>
/* javascript code goes here... */
</script>
}}}
All parameters are //optional//. When the ''show'' keyword is used, the plugin will also include the script source code in the output that it displays in the tiddler. This is helpful when creating examples for documentation purposes (such as used in this tiddler!)
__''Deferred execution from an 'onClick' link''__
<script label="click here" title="mouseover tooltip text" key="X" show>
/* javascript code goes here... */
alert('you clicked on the link!');
</script>
By including a {{{label="..."}}} parameter in the initial {{{<script>}}} marker, the plugin will create a link to an 'onclick' script that will only be executed when that specific link is clicked, rather than running the script each time the tiddler is rendered. You may also include a {{{title="..."}}} parameter to specify the 'tooltip' text that will appear whenever the mouse is moved over the onClick link text, and a {{{key="X"}}} parameter to specify an //access key// (which must be a //single// letter or numeric digit only).
__''Loading scripts from external source files''__
<script src="URL" show>
/* optional javascript code goes here... */
</script>You can also load javascript directly from an external source URL, by including a src="..." parameter in the initial {{{<script>}}} marker (e.g., {{{<script src="demo.js"></script>}}}). This is particularly useful when incorporating third-party javascript libraries for use in custom extensions and plugins. The 'foreign' javascript code remains isolated in a separate file that can be easily replaced whenever an updated library file becomes available.
In addition to loading the javascript from the external file, you can also use this feature to invoke javascript code contained within the {{{<script>...</script>}}} markers. This code is invoked //after// the external script file has been processed, and can make immediate use of the functions and/or global variables defined by the external script file.
>Note: To ensure that your javascript functions are always available when needed, you should load the libraries from a tiddler that is rendered as soon as your TiddlyWiki document is opened, such as MainMenu. For example: put your {{{<script src="..."></script>}}} syntax into a separate 'library' tiddler (e.g., LoadScripts), and then add {{{<<tiddler LoadScripts>>}}} to MainMenu so that the library is loaded before any other tiddlers that rely upon the functions it defines.
>
>Normally, loading external javascript in this way does not produce any direct output, and should not have any impact on the appearance of your MainMenu. However, if your LoadScripts tiddler contains notes or other visible content, you can suppress this output by using 'inline CSS' in the MainMenu, like this: {{{@@display:none;<<tiddler LoadScripts>>@@}}}
<<<
!!!!!Creating dynamic tiddler content and accessing the ~TiddlyWiki DOM
<<<
An important difference between TiddlyWiki inline scripting and conventional embedded javascript techniques for web pages is the method used to produce output that is dynamically inserted into the document: in a typical web document, you use the {{{document.write()}}} (or {{{document.writeln()}}}) function to output text sequences (often containing HTML tags) that are then rendered when the entire document is first loaded into the browser window.
However, in a ~TiddlyWiki document, tiddlers (and other DOM elements) are created, deleted, and rendered "on-the-fly", so writing directly to the global 'document' object does not produce the results you want (i.e., replacing the embedded script within the tiddler content), and instead will //completely replace the entire ~TiddlyWiki document in your browser window (which is clearly not a good thing!)//. In order to allow scripts to use {{{document.write()}}}, the plugin automatically converts and buffers all HTML output so it can be safely inserted into your tiddler content, immediately following the script.
''Note that {{{document.write()}}} can only be used to output "pure HTML" syntax. To produce //wiki-formatted// output, your script should instead return a text value containing the desired wiki-syntax content'', which will then be automatically rendered immediately following the script. If returning a text value is not sufficient for your needs, the plugin also provides an automatically-defined variable, 'place', that gives the script code ''direct access to the //containing DOM element//'' into which the tiddler output is being rendered. You can use this variable to ''perform direct DOM manipulations'' that can, for example:
* generate wiki-formatted output using {{{wikify("...content...",place)}}}
* vary the script's actions based upon the DOM element in which it is embedded
* access 'tiddler-relative' DOM information using {{{story.findContainingTiddler(place)}}}
Note:
''When using an 'onclick' script, the 'place' element actually refers to the onclick //link text// itself, instead of the containing DOM element.'' This permits you to directly reference or modify the link text to reflect any 'stateful' conditions that might set by the script. To refer to the containing DOM element from within an 'onclick' script, you can use "place.parentNode" instead.
<<<
!!!!!Instant "bookmarklets"
<<<
You can also use an 'onclick' link to define a "bookmarklet": a small piece of javascript that can be ''invoked directly from the browser without having to be defined within the current document.'' This allows you to create 'stand-alone' commands that can be applied to virtually ANY TiddlyWiki document... even remotely-hosted documents that have been written by others!! To create a bookmarklet, simply define an 'onclick' script and then grab the resulting link text and drag-and-drop it onto your browser's toolbar (or right-click and use the 'bookmark this link' command to add it to the browser's menu).
Notes:
*When writing scripts intended for use as bookmarklets, due to the ~URI-encoding required by the browser, ''you cannot not use ANY double-quotes (") within the bookmarklet script code.''
*All comments embedded in the bookmarklet script must ''use the fully-delimited {{{/* ... */}}} comment syntax,'' rather than the shorter {{{//}}} comment syntax.
*Most importantly, because bookmarklets are invoked directly from the browser interface and are not embedded within the TiddlyWiki document, there is NO containing 'place' DOM element surrounding the script. As a result, ''you cannot use a bookmarklet to generate dynamic output in your document,'' and using {{{document.write()}}} or returning wiki-syntax text or making reference to the 'place' DOM element will halt the script and report a "Reference Error" when that bookmarklet is invoked.
Please see [[InstantBookmarklets]] for many examples of 'onclick' scripts that can also be used as bookmarklets.
<<<
!!!!!Special reserved function name
<<<
The plugin 'wraps' all inline javascript code inside a function, {{{_out()}}}, so that any return value you provide can be correctly handled by the plugin and inserted into the tiddler. To avoid unpredictable results (and possibly fatal execution errors), this function should never be redefined or called from ''within'' your script code.
<<<
!!!!!$(...) 'shorthand' function
<<<
As described by Dustin Diaz [[here|http://www.dustindiaz.com/top-ten-javascript/]], the plugin defines a 'shorthand' function that allows you to write:
{{{
$(id)
}}}
in place of the normal standard javascript syntax:
{{{
document.getElementById(id)
}}}
This function is provided merely as a convenience for javascript coders that may be familiar with this abbreviation, in order to allow them to save a few bytes when writing their own inline script code.
<<<
!!!!!Examples
<<<
simple dynamic output:
><script show>
document.write("The current date/time is: "+(new Date())+"<br>");
return "link to current user: [["+config.options.txtUserName+"]]\n";
</script>
dynamic output using 'place' to get size information for current tiddler:
><script show>
if (!window.story) window.story=window;
var title=story.findContainingTiddler(place).getAttribute("tiddler");
var size=store.getTiddlerText(title).length;
return title+" is using "+size+" bytes";
</script>
dynamic output from an 'onclick' script, using {{{document.write()}}} and/or {{{return "..."}}}
><script label="click here" show>
document.write("<br>The current date/time is: "+(new Date())+"<br>");
return "link to current user: [["+config.options.txtUserName+"]]\n";
</script>
creating an 'onclick' button/link that accesses the link text AND the containing tiddler:
><script label="click here" title="clicking this link will show an 'alert' box" key="H" show>
if (!window.story) window.story=window;
var txt=place.firstChild.data;
var tid=story.findContainingTiddler(place).getAttribute('tiddler');
alert('Hello World!\nlinktext='+txt+'\ntiddler='+tid);
</script>
dynamically setting onclick link text based on stateful information:
>{{block{
{{{
<script label="click here">
/* toggle "txtSomething" value */
var on=(config.txtSomething=="ON");
place.innerHTML=on?"enable":"disable";
config.txtSomething=on?"OFF":"ON";
return "\nThe current value is: "+config.txtSomething;
</script><script>
/* initialize onclick link text based on current "txtSomething" value */
var on=(config.txtSomething=="ON");
place.lastChild.previousSibling.innerHTML=on?"disable":"enable";
</script>
}}}
<script label="click here">
/* toggle "txtSomething" value */
var on=(config.txtSomething=="ON");
place.innerHTML=on?"enable":"disable";
config.txtSomething=on?"OFF":"ON";
return "\nThe current value is: "+config.txtSomething;
</script><script>
/* initialize onclick link text based on current "txtSomething" value */
var on=(config.txtSomething=="ON");
place.lastChild.innerHTML=on?"enable":"disable";
</script>
}}}
loading a script from a source url:
>http://www.TiddlyTools.com/demo.js contains:
>>{{{function inlineJavascriptDemo() { alert('Hello from demo.js!!') } }}}
>>{{{displayMessage('InlineJavascriptPlugin: demo.js has been loaded');}}}
>note: When using this example on your local system, you will need to download the external script file from the above URL and install it into the same directory as your document.
>
><script src="demo.js" show>
return "inlineJavascriptDemo() function has been defined"
</script>
><script label="click to invoke inlineJavascriptDemo()" key="D" show>
inlineJavascriptDemo();
</script>
<<<
!!!!!Revisions
<<<
2010.12.15 1.9.6 allow (but ignore) type="..." syntax
2009.04.11 1.9.5 pass current tiddler object into wrapper code so it can be referenced from within 'onclick' scripts
2009.02.26 1.9.4 in $(), handle leading '#' on ID for compatibility with JQuery syntax
2008.06.11 1.9.3 added $(...) function as 'shorthand' for document.getElementById()
2008.03.03 1.9.2 corrected fallback declaration of wikifyPlainText() (fixes Safari "parse error")
2008.02.23 1.9.1 in onclick function, use string instead of array for 'bufferedHTML' (fixes IE errors)
2008.02.21 1.9.0 output from 'onclick' scripts (return value or document.write() calls) are now buffered and rendered into into a span following the script. Also, added default 'return false' handling if no return value provided (prevents HREF from being triggered -- return TRUE to allow HREF to be processed). Thanks to Xavier Verges for suggestion and preliminary code.
2008.02.14 1.8.1 added backward-compatibility for use of wikifyPlainText() in TW2.1.3 and earlier
2008.01.08 [*.*.*] plugin size reduction: documentation moved to ...Info tiddler
2007.12.28 1.8.0 added support for key="X" syntax to specify custom access key definitions
2007.12.15 1.7.0 autogenerate URI encoded HREF on links for onclick scripts. Drag links to browser toolbar to create bookmarklets. IMPORTANT NOTE: place is NOT defined when scripts are used as bookmarklets. In addition, double-quotes will cause syntax errors. Thanks to PaulReiber for debugging and brainstorming.
2007.11.26 1.6.2 when converting "document.write()" function calls in inline code, allow whitespace between "write" and "(" so that "document.write ( foobar )" is properly converted.
2007.11.16 1.6.1 when rendering "onclick scripts", pass label text through wikifyPlainText() to parse any embedded wiki-syntax to enable use of HTML entities or even TW macros to generate dynamic label text.
2007.02.19 1.6.0 added support for title="..." to specify mouseover tooltip when using an onclick (label="...") script
2006.10.16 1.5.2 add newline before closing '}' in 'function out_' wrapper. Fixes error caused when last line of script is a comment.
2006.06.01 1.5.1 when calling wikify() on script return value, pass hightlightRegExp and tiddler params so macros that rely on these values can render properly
2006.04.19 1.5.0 added 'show' parameter to force display of javascript source code in tiddler output
2006.01.05 1.4.0 added support 'onclick' scripts. When label="..." param is present, a button/link is created using the indicated label text, and the script is only executed when the button/link is clicked. 'place' value is set to match the clicked button/link element.
2005.12.13 1.3.1 when catching eval error in IE, e.description contains the error text, instead of e.toString(). Fixed error reporting so IE shows the correct response text. Based on a suggestion by UdoBorkowski
2005.11.09 1.3.0 for 'inline' scripts (i.e., not scripts loaded with src="..."), automatically replace calls to 'document.write()' with 'place.innerHTML+=' so script output is directed into tiddler content. Based on a suggestion by BradleyMeck
2005.11.08 1.2.0 handle loading of javascript from an external URL via src="..." syntax
2005.11.08 1.1.0 pass 'place' param into scripts to provide direct DOM access
2005.11.08 1.0.0 initial release
<<<
!Input
Les différents formats d'entrée rencontrés sont :
* //raw// text : texte brut <<tag in:raw>>
* //tokenized// text :
** <<tag in:sent>> 1 //sentence// par ligne, //token//s (y compris les ponctuations) separés par des //espace//s <<slider show-in-sent-ex [[exemple in:sent]] "(exemple)" "exemple d'entrée tokenisée 1 phrase par ligne.">>
** <<tag in:tok>> 1 //token// par ligne, avec ou sans marque de fin de phrase (le plus souvent une ligne blanche)
** <<tag in:cols>> 1 //token// par ligne, multicolones
* //xml// : <<tag in:xml>>
** [[Stanford POS Tagger]] : extraction du texte brut contenu sous une liste de noeuds => //raw// text.
Du point de vue //informationnel//, il y a trois niveaux d'entrée :
* texte ''brut'', i.e. sans informations supplémentaires (entrées étiquetées <<tag in:raw>>, mais aussi l'entrée XML du [[Stanford POS Tagger]])
* texte ''tokenisé'', i.e. délimitation des tokens, et parfois également des phrases (entrées étiquetées <<tag in:sent>> ou <<tag in:tok>>)
* texte ''tokenisé enrichi'', i.e. délimitation des tokens et informations additionnelles (entrées multicolones <<tag in:cols>>)
Du point de vue du //format//, la variabilité est plus grande :
* différents ''encoding'' : pour le moins UTF-8, latin1(ISO8859-1 et ISO8859-15) + systèmes windows/mac pour le français (+ absence d'accents)
* signe de ''fin de ligne'' : dos(CRLF) / unix(LF) / mac(CR) -- normalement BufferedReader.readLine() gère les 3 cas
* dans le cas du ''texte brut'', la marque de ''fin de ligne'' peut-être interprétée comme :
** coupure régulière du texte (//wrap//), auquel cas il faut joindre les lignes __sans nécessairement insérer d'espace__
** marque de fin de ligne (entrée 1 phrase par ligne)
** marque de fin de paragraphe/section, auquel cas une ligne peut contenir plusieurs phrases.
[img[http://www.loria.fr/logo.png]] jSafran : Free syntax editor/parser
Christophe Cerisara, LORIA
//Installeur, code et démo actuellement indisponible//
* Intégration de différents outils : TreeTagger, MaltParser, MATE parser
* Interface graphique
<<<
JSafran is an open-source 100%-Java software to annotate a text&audio corpus with syntactic dependency trees, either manually, automatically or semi-automatically. It integrates the Malt-parser, the MATE parser, the Treetagger, plus experimental unsupervised rule-base Bayesian models. It can be interfaced with JTrans to automatically align the sound file and listen to specific utterances. It integrates JGIT to support collaborative edition & versionning, CoNLL'05-06-08-09 I/O, and many more features...
<<<
http://www-igm.univ-mlv.fr/~mconstan/index.php?section=software
Matthieu Constant : [img[http://igm.univ-mlv.fr/LIGM/images/LIGM.png]] LIGM (Laboratoire d'informatique Gaspard-Monge, Université Paris-Est Marne-la-Vallée)
<<<
LGTagger is an open-source Part-of-speech tagger that also recognizes Multiword units. It is based on Conditional Random Fields (CRF) and large-coverage lexical resources. The lexical resources can be composed of morphosyntactic dictionaries (including simple and compound words) and strongly lexicalized local grammars. It presently works for French.
Reference: Matthieu Constant and Anthony Sigogne. ~MWU-aware ~Part-of-Speech Tagging with a CRF model and lexical resources. ACL Workshop on Multiword Expressions: from Parsing and Generation to the Real World (MWE'11). 2011. [[pdf|http://hal-upec-upem.archives-ouvertes.fr/docs/00/62/15/85/PDF/constant_sigogne.pdf]]
<<<
Langage : Java + librairies/outils en C/C++ (Unitex + Wapiti pour les CRF)
Licence LGPL & resources LGPL-LR
;Input : utf-8
:- tokenized and segmented in sentences (one token per line, blank line to separate sentences)
:- raw ⇒ (1) sentence segmented with Unitext for French (2) token separed by regex (sequences of letters + a few particular cases)
;Tag set :
:[[CC Tagset]] (Crabbé & Candito, 2008)
!!Algorithme
!!! POS tagging
Utilise un taggeur CRF.
Construit un transducteur acyclique (TFST) dont les transistions sont les tokens associés à chaque POS tag, puis cherche le chemin optimum.
Features :
* //Internal unigram features//
** w~~0~~ (form)
** Lowercase form of w~~0~~
** Prefix of w~~0~~ = P with |P| < 5
** Suffix of w~~0~~ = S with |S| < 5
** w~~0~~ contains a hyphen
** w~~0~~ contains a digit
** w~~0~~ is capitalized
** w~~0~~ is all capital
** w~~0~~ is capitalized and BOS (Begin Of Sentence)
** w~~0~~ is multiword
** Lexicon tags AC~~0~~ of w~~0~~ & w~~0~~ is multiword
* //Contextual unigram features//
** w~~i~~ i ∈ {−2, −1, 1, 2}
** w~~i~~ w~~j~~ , (j, k) ∈ {(−1, 0), (0, 1), (−1, 1)}
** AC~~i~~ & w~~i~~ is multiword, i ∈ {−2, −1, 1, 2}
* Bigram features
** t~~−1~~ (POS tag précédent)
nota : //Ambiguity Class of the token// (AC) : liste de POS candidats trouvés dans un lexique externe.
@@(!)@@ les ressources lexicales ont 2 fonctions :
* définir la //feature// AC pour le taggeur CRF
* réduire la liste des POS candidats (i.e. pour un token dans le lexique, seuls les transitions avec un POS tag du lexique sont conservés avant de chercher le chemin optimum).
!!!Pour les MWE :
* Le taggeur est entrainé avec les POS couplés avec une étiquette //"IOB"// (__B__egin, __I__nside, __O__utside : POS+B, POS+I ou POS)
* Des ressources lexicales variées permettent d'ajouter des //features// d'apprentissage :
** POS : POS de l'unité multiword (ex: "Banque de Chine" NPP, "pouvoir d'achat" NC)
** STRUCT : (optionnel) structure interne (ex "pouvoir d'achat" NPN=Noun Preposition Noun)
** SEM : (optionnel) sémantique (ex "Banque de Chine" ORG=organisation)
** POSITION : Begin/Inside/Outside
* Ajout des transitions correspondants aux MWE
!! Ressources lexicales
* Dictionnaires morphosyntaxiques
** DELA ([[téléchargement|http://infolingu.univ-mlv.fr/DonneesLinguistiques/Dictionnaires/telechargement.html]])
** Lefff ([[http://www.labri.fr/perso/clement/lefff/]])
** Prolex (toponymes) ([[http://www.cnrtl.fr/lexiques/prolex/]])
** Organizations (dicctionaire d'organisation)
** First names (dictionnaire de prénoms)
* Grammaires locales
** grammaires faites manuellement à partir de celles trouvées sur [[GraalWeb|http://igm.univ-mlv.fr/~mconstan/library/index_graalweb.html]]
[img(100px+,)[http://lia.univ-avignon.fr/sites/default/files/pictures/LogoLIA4.png]] LIA_TAGG: a statistical POS tagger + syntactic bracketer
Dévellopé par F. Béchet au LIA : http://lia.univ-avignon.fr/fileadmin/documents/Users/Intranet/chercheurs/bechet/download_fred.html
Aussi sur sa page perso au LIF : http://pageperso.lif.univ-mrs.fr/~frederic.bechet/download.html
Pas de documentation en ligne.
Fichier //README// joint au code source.
;Input : (''ISO8859-1'')
- tokenized text, 1 token/line (? balises de phrase {{{<s> </s>}}})
- le texte brut peut être préparé par l'outil //lia_clean//
;Output : (''ISO8859-1'')
- 1 token/line {{{WORD POS}}} (pour le français, il existe une sortie avec le {{{LEMMA}}})
Note : option pour __réaccentuer le texte__
[img[http://www.languagetool.org/images/LanguageToolBig.png]]http://www.languagetool.org/
> LanguageTool is an Open Source proofreading software for English, French, German, Polish, Romanian, and [[more than 20 other languages|http://www.languagetool.org/languages/]]. It finds many errors that a simple spell checker cannot detect like mixing up there/their and it detects some grammar problems.
Tips and Tricks : (http://wiki.languagetool.org/tips-and-tricks)
<<<
Tagging a corpus using LanguageTool
LanguageTool has a POS tagger, sometimes it has a disambiguator or a chunker for a language, so you can use it to tag a big corpus. We added a special command-line switch —taggeronly or, for short, -t to disable rule checking and use only the tagger.
We didn't go beyond 3 GB of pure text but you should experience no problem even with huge corpora.
{{{java -jar LanguageTool.jar -l <language> -c <encoding> -t <corpus_file> > <tagged_corpus_file>}}}
<<<
;Input :
:raw text
;Output :
:append inside brackets lemma/POS proposition__s__ (comma separated) to each token
:+ sentence tags {{{<S> </S>}}}
:+ /!\ 2 initial lines of information in the output : <<slider show-ltpos-out-start [[languagetool output start]] "(show/hide)" "example of the 2 firts lines of information in LanguageTool POS Tagger.">>
: {{{<S> This[this/DT] is[be/VBZ] a[a/DT] sample[sample/JJ,sample/NN,sample/VB,sample/VBP] sentence[sentence/NN,sentence/VB,sentence/VBP].[./.,</S>]}}}
;Tag set : (in source code)
:- EN: Derived from [[Penn Treebank English POS tag set]] see [[LanguageTool English Tagset]]
:- FR: see [[LanguageTool French Tagset]]
File {{{languagetool-language-modules/en/src/main/resources/org/languagetool/resource/en/tagset.txt}}} in the source code :
{{{
These are mostly the tags of the Penn Treebank tagset as used by LanguageTool,
with examples. See "new tag" for tags introduced by LanguageTool.
For more details, also see http://www.cis.upenn.edu/~treebank/
CC Coordinating conjunction: and, or, either, if, as, since, once, neither, less
CD Cardinal number: one, two, twenty-four
DT Determiner: an, an, all, many, much, any, some, this
EX Existential there: there (no other words)
FW Foreign word: infinitum, ipso
IN Preposition/subordinate conjunction: except, inside, across, on, through, beyond, with, without
JJ Adjective: beautiful, large, inspectable
JJR Adjective, comparative: larger, quicker
JJS Adjective, superlative: largest, quickest
LS List item marker: not used by LanguageTool
MD Modal: should, can, need, must, will, would
NN Noun, singular or mass: bicycle, earthquake, zipper
NNS Noun, plural: bicycles, earthquake, zippers
NN:U Mass noun #new tag - deviation from Penn: admiration, air, Afrikaans
NN:UN Noun used as mass #new tag - deviation from Penn: establishment, wax, afternoon
NNP Proper noun, singular: Denver, DORAN, Alexandra
NNPS Proper noun, plural: Buddhists, Englishmen
PDT Predeterminer: all, sure, such, this, many, half, both, quite
POS Possessive ending: s (as in: Peter's)
PRP Personal pronoun: everyone, I, he, it, myself
PRP$ Possessive pronoun: its, our, their, mine, my, her, his, your
RB Adverb and negation: easily, sunnily, suddenly, specifically, not
RBR Adverb, comparative: better, faster, quicker
RBS Adverb, superlative: best, fastest, quickest
RP Particle: in, into, at, off, over, by, for, under
SYM Symbol: not used by LanguageTool
TO to: to (no other words)
UH Interjection: aargh, ahem, attention, congrats, help
VB Verb, base form: eat, jump, believe
VBD Verb, past tense: ate, jumped, believed
VBG Verb, gerund/present participle: eating, jumping, believing
VBN Verb, past participle: eaten, jumped, believed
VBP Verb, non-3rd ps. sing. present: eat, jump, believe
VBZ Verb, 3rd ps. sing. present: eats, jumps, believes
WDT wh-determiner: that, whatever, what, whichever, which (no other words)
WP wh-pronoun: that, whatever, what, whatsoever, whosoever, who, whom, whoever, which (no other words)
WP$ Possessive wh-pronoun: whose (no other words)
WRB wh-adverb: however, how, whereever, where, when, why
`` Left open double quote
, Comma
'' Right close double quote
. Sentence-final punctuation (in LanguageTool, use SENT_END instead)
: Colon, semi-colon
$ Dollar sign
# Pound sign
}}}
Tagset used by : <<list filter [tag[tags:LT-FR]]>>
(?) Connection with Dicollecte POS : http://www.dicollecte.org/documentation.php?prj=fr#grammaire ?
file {{{languagetool-language-modules/fr/src/main/resources/org/languagetool/resource/fr/tagset.LT.txt}}} in the source code :
{{{
TAGS FOR LANGUAGETOOL
http://www.languagetool.org/
http://community.languagetool.org/
http://languagetool.wikidot.com/
_______________________________________________________________________________
-- NOUNS AND ADJECTIVES --
Noun: N [gender] [number]
Adjective: J [gender] [number]
[gender] m = masculine
f = feminine
e = epicene
[number] s = singular
p = plural
sp = singular or plural
Examples:
N m s
N f p
J e sp
-- MISC --
Adverb: A
Interjection: I
Onomatopeia: O
Cardinal number: K
Abbreviation: S
Proper name: Z
Marker M
-- VERBS --
Infinitive: V inf
Conjugation: V [mood] [tense] [person] [number]
Present participle: V ppr
Past participle: V ppa [gender] [number]
[mood] ind = indicative
con = conditional
sub = subjonctive
imp = imperative
[tense] pres = present
psim = “passé simple” (past: action done once)
impa = “imparfait” (past: action done regularly)
futu = future
[person] 1 = first person
2 = second person
3 = third person
[number] s = singular
p = plural
Exceptions:
avoir “V avoir” instead of “V”
être “V etre” instead of “V”
Examples:
V ind pres 1 p
V ind psim 3 p
V sub impa 3 s
V inf
V ppa f p
-- GRAMMATICAL WORDS --
preposition: P
conjonction: C
subordinating conjunction: C sub
coordinating conjunction: C coor
determiner: D
D [gender] [number]
pronoun: R [gender] [number]
relative pronoun: R rel [gender] [number]
demonstrative pronoun: R dem [gender] [number]
reflexive pronoun: R refl [person] [gender] [number]
personal pronoun: R pers [to] [gender] [number]
R pers [to] [number]
[gender] m = masculine
f = feminine
e = epicene
[number] s = singular
p = plural
[person] 1 = first person
2 = second person
3 = third person
[to] suj = subject
obj = object
}}}
/***
|Name|ListTagsPlugin|
|Source|derived from http://www.TiddlyTools.com/#MatchTagsPlugin|
|Documentation|@@TODO@@|
|Version|0 (2.0.6)|
|Author|GMM (Eric Shulman)|
|License|http://www.TiddlyTools.com/#LegalStatements|
|~CoreVersion|2.1|
|Type|plugin|
|Description|list of tags with full boolean expressions (AND, OR, NOT, nested parentheses, and regex)|
!!!!!Documentation
> Something like [[MatchTagsPluginInfo]] but create list of tags
> Define the macro //listTags//
> Only the //inline// mode is implemented
<<<
Format take 3 variables :
* %0 : tag
* %1 : tag macro with the tag (i.e. {{{<<tag [[%0]]>>}}})
* %2 : {{{>>}}} (to use other macros, by example {{{"<<tag [[%0]]%2"}}} is equivalent of {{{"%1"}}})
<<<
!!!!!Code
***/
//{{{
version.extensions.ListTagsPlugin= {major: 0, minor: 2, revision: 0, date: new Date(2013,08,23)};
// store.getMatchingTags() processes boolean expressions for tag matching
// sortfield (optional) sets sort order for tiddlers - default=title
// tiddlers (optional) use alternative set of tiddlers (instead of current store)
TiddlyWiki.prototype.getMatchingTags = function(tagexpr,sortfield,tiddlers) {
var debug=config.options.chkDebug; // abbreviation
var cmm=config.macros.listTags; // abbreviation
var r=[]; // results are an array of tiddlers
var tids=tiddlers||store.getTiddlers();
if (tids && sortfield) tids=store.sortTiddlers(tids,sortfield);
if (debug) displayMessage(cmm.msg1.format([tids.length]));
// convert expression into javascript code with regexp tests,
// so that "tag1 AND ( tag2 OR NOT tag3 )" becomes
// "/\~tag1\~/.test(...) && ( /\~tag2\~/.test(...) || ! /\~tag3\~/.test(...) )"
// normalize whitespace, tokenize operators, delimit with "~"
var c=tagexpr.trim(); // remove leading/trailing spaces
c = c.replace(/\s+/ig," "); // reduce multiple spaces to single spaces
c = c.replace(/\(\s?/ig,"~(~"); // open parens
c = c.replace(/\s?\)/ig,"~)~"); // close parens
c = c.replace(/(\s|~)?&&(\s|~)?/ig,"~&&~"); // &&
c = c.replace(/(\s|~)AND(\s|~)/ig,"~&&~"); // AND
c = c.replace(/(\s|~)?\|\|(\s|~)?/ig,"~||~"); // ||
c = c.replace(/(\s|~)OR(\s|~)/ig,"~||~"); // OR
c = c.replace(/(\s|~)?!(\s|~)?/ig,"~!~"); // !
c = c.replace(/(^|~|\s)NOT(\s|~)/ig,"~!~"); // NOT
c = c.replace(/(^|~|\s)NOT~\(/ig,"~!~("); // NOT(
// change tag terms to regexp tests
var terms=c.split("~"); for (var i=0; i<terms.length; i++) { var t=terms[i];
if (/(&&)|(\|\|)|[!\(\)]/.test(t) || t=="") continue; // skip operators/parens/spaces
terms[i]="/^"+t+"$/.test(tiddlertag)";
}
c=terms.join(" ");
if (debug) { displayMessage(cmm.msg2.format([tagexpr])); displayMessage(cmm.msg3.format([c])); }
// scan tiddlers for matches
for (var t=0; t<tids.length; t++) {
for (var tt=0; tt<tids[t].tags.length; tt++) {
// assemble tags from tiddler into string "~tag1~tag2~tag3~"
var tiddlertag = tids[t].tags[tt];
if (r.indexOf(tiddlertag) >=0 ) continue;
try { if(eval(c)) r.push(tiddlertag); } // test tags
catch(e) { // error in test
displayMessage(cmm.msg2.format([tagexpr]));
displayMessage(cmm.msg3.format([c]));
displayMessage(e.toString());
break; // skip remaining tiddlers
}
}
}
if (debug) displayMessage(cmm.msg4.format([r.length,tagexpr]));
return r;
}
//}}}
//{{{
config.macros.listTags = {
msg1: "scanning %0 input tiddlers",
msg2: "looking for '%0'",
msg3: "using expression: '%0'",
msg4: "found %0 tiddlers matching '%1'",
noMatch: "no matching tiddlers",
untaggedKeyword: "-",
untaggedLabel: "no tags",
untaggedPrompt: "show tiddlers with no tags",
defTiddler: "MatchingTiddlers",
defTags: "",
defFormat: "%0",
defSeparator: "\n",
reportHeading: "Found %0 tiddlers tagged with: '{{{%1}}}'\n----\n",
handler: function(place,macroName,params,wikifier,paramString,tiddler) {
var mode=params[0]?params[0].toLowerCase():'';
if (mode=="inline")
params.shift();
if (mode=="report" || mode=="panel") {
params.shift();
var target=params.shift()||this.defTiddler;
}
if (mode=="popup") {
params.shift();
if (params[0]&¶ms[0].substr(0,6)=="label:") var label=params.shift().substr(6);
if (params[0]&¶ms[0].substr(0,7)=="prompt:") var prompt=params.shift().substr(7);
} else {
var fmt=(params.shift()||this.defFormat).unescapeLineBreaks();
var sep=(params.shift()||this.defSeparator).unescapeLineBreaks();
}
var sortBy="+title";
if (params[0]&¶ms[0].substr(0,5)=="sort:") sortBy=params.shift().substr(5);
var expr = params.join(" ");
if (mode!="panel" && (!expr||!expr.trim().length)) return;
if (expr==this.untaggedKeyword)
{ var label=this.untaggedLabel; var prompt=this.untaggedPrompt };
switch (mode) {
case "popup": this.createPopup(place,label,expr,prompt,sortBy); break;
case "panel": this.createPanel(place,expr,fmt,sep,sortBy,target); break;
case "report": this.createReport(target,this.defTags,expr,fmt,sep,sortBy); break;
case "inline": default: this.createInline(place,expr,fmt,sep,sortBy); break;
}
},
formatList: function(tags,fmt,sep) {
var out=[];
for (var i=0; i<tags.length; i++) { var t=tags[i];
var title=t;//t.title;
var tagmacro="<<tag [["+t+"]]>>";
//var who=t.modifier;
//var when=t.modified.toLocaleString();
//var text=t.text;
//var first=t.text.split("\n")[0];
//var desc=store.getTiddlerSlice(t.title,"description");
//desc=desc||store.getTiddlerSlice(t.title,"Description");
//desc=desc||store.getTiddlerText(t.title+"##description");
//desc=desc||store.getTiddlerText(t.title+"##Description");
//var tags=t.tags.length?'[['+t.tags.join(']] [[')+']]':'';
//out.push(fmt.format([title,who,when,text,first,desc,tags]));
out.push(fmt.format([title, tagmacro, ">>"]));
}
return out.join(sep);
},
createInline: function(place,expr,fmt,sep,sortBy) {
wikify(this.formatList(store.getMatchingTags(expr),fmt,sep),place);
},
createPopup: function(place,label,expr,prompt,sortBy) {
var btn=createTiddlyButton(place,
(label||expr).format([expr]),
(prompt||config.views.wikified.tag.tooltip).format([expr]),
function(ev){ return config.macros.listTags.showPopup(this,ev||window.event); });
btn.setAttribute("sortBy",sortBy);
btn.setAttribute("expr",expr);
},
showPopup: function(here,ev) {
var p=Popup.create(here,null,"listTags popup"); if (!p) return false;
var tids=store.getMatchingTags(here.getAttribute("expr"));
store.sortTiddlers(tids,here.getAttribute("sortBy"));
var list=[]; for (var t=0; t<tids.length; t++) list.push(tids[t].title);
if (!list.length) createTiddlyText(p,this.noMatch);
else {
var b=createTiddlyButton(createTiddlyElement(p,"li"),
config.views.wikified.tag.openAllText,
config.views.wikified.tag.openAllTooltip,
function() {
var list=this.getAttribute("list").readBracketedList();
story.displayTiddlers(null,tids);
});
b.setAttribute("list","[["+list.join("]] [[")+"]]");
createTiddlyElement(p,"hr");
}
var out=this.formatList(tids," [[%0]] ","\n"); wikify(out,p);
Popup.show();
ev.cancelBubble=true;
if(ev.stopPropagation) ev.stopPropagation();
return false;
},
createReport: function(target,tags,expr,fmt,sep,sortBy) {
var tids=store.sortTiddlers(store.getMatchingTags(expr),sortBy);
if (!tids.length) { displayMessage('no matches for: '+expr); return false; }
var msg=config.messages.overwriteWarning.format([target]);
if (store.tiddlerExists(target) && !confirm(msg)) return false;
var out=this.reportHeading.format([tids.length,expr])
out+=this.formatList(tids,fmt,sep);
store.saveTiddler(target,target,out,config.options.txtUserName,new Date(),tags,{});
story.closeTiddler(target); story.displayTiddler(null,target);
},
createPanel: function(place,expr,fmt,sep,sortBy,tid) {
var s=createTiddlyElement(place,"span"); s.innerHTML=store.getTiddlerText("ListTagsPlugin##html");
var f=s.getElementsByTagName("form")[0];
f.expr.value=expr; f.fmt.value=fmt; f.sep.value=sep.escapeLineBreaks();
f.tid.value=tid; f.tags.value=this.defTags;
}
};
//}}}
/***
//{{{
!html
<form style='display:inline;white-space:nowrap'>
<input type='text' name='expr' style='width:50%' title='tag expression'><!--
--><input type='text' name='fmt' style='width:10%' title='list item format'><!--
--><input type='text' name='sep' style='width:5%' title='list item separator'><!--
--><input type='text' name='tid' style='width:12%' title='target tiddler title'><!--
--><input type='text' name='tags' style='width:10%' title='target tiddler tags'><!--
--><input type='button' name='go' style='width:8%' value='go' onclick="
var expr=this.form.expr.value;
if (!expr.length) { alert('Enter a boolean tag expression'); return false; }
var fmt=this.form.fmt.value;
if (!fmt.length) { alert('Enter the list item output format'); return false; }
var sep=this.form.sep.value.unescapeLineBreaks();
var tid=this.form.tid.value;
if (!tid.length) { alert('Enter a target tiddler title'); return false; }
var tags=this.form.tags.value;
config.macros.listTags.createReport(tid,tags,expr,fmt,sep,'title');
return false;">
</form>
!end
//}}}
***/
//{{{
// SHADOW TIDDLER for displaying default panel input form
config.shadowTiddlers.MatchTags="<<listTags panel>>";
//}}}
//{{{
// TWEAK core filterTiddlers() or config.filters['tag'] (in TW262+)
// to use getMatchingTiddlers instead getTaggedTiddlers
// for enhanced boolean matching in [tag[...]] syntax
var TW262=config.filters && config.filters['tag']; // detect TW262+
var fname=TW262?"config.filters['tag']":"TiddlyWiki.prototype.filterTiddlers";
var code=eval(fname).toString().replace(/getTaggedTiddlers/g,'getMatchingTiddlers');
eval(fname+'='+code);
//}}}
//{{{
// REDEFINE core handler for enhanced boolean matching in tag:"..." paramifier
// use filterTiddlers() instead of getTaggedTiddlers() to get list of tiddlers.
config.paramifiers.tag = {
onstart: function(v) {
var tagged = store.filterTiddlers("[tag["+v+"]]");
story.displayTiddlers(null,tagged,null,false,null);
}
};
//}}}
/***
|''Name:''|LoadRemoteFileThroughProxy (previous LoadRemoteFileHijack)|
|''Description:''|When the TiddlyWiki file is located on the web (view over http) the content of [[SiteProxy]] tiddler is added in front of the file url. If [[SiteProxy]] does not exist "/proxy/" is added. |
|''Version:''|1.1.0|
|''Date:''|mar 17, 2007|
|''Source:''|http://tiddlywiki.bidix.info/#LoadRemoteFileHijack|
|''Author:''|BidiX (BidiX (at) bidix (dot) info)|
|''License:''|[[BSD open source license|http://tiddlywiki.bidix.info/#%5B%5BBSD%20open%20source%20license%5D%5D ]]|
|''~CoreVersion:''|2.2.0|
***/
//{{{
version.extensions.LoadRemoteFileThroughProxy = {
major: 1, minor: 1, revision: 0,
date: new Date("mar 17, 2007"),
source: "http://tiddlywiki.bidix.info/#LoadRemoteFileThroughProxy"};
if (!window.bidix) window.bidix = {}; // bidix namespace
if (!bidix.core) bidix.core = {};
bidix.core.loadRemoteFile = loadRemoteFile;
loadRemoteFile = function(url,callback,params)
{
if ((document.location.toString().substr(0,4) == "http") && (url.substr(0,4) == "http")){
url = store.getTiddlerText("SiteProxy", "/proxy/") + url;
}
return bidix.core.loadRemoteFile(url,callback,params);
}
//}}}
[img[http://alpage.inria.fr/statgram/frdep/I/alpage.png]] http://lingwb.gforge.inria.fr/
;Input : utf-8
:- tokenized and segmented in sentences (__one sentence per line__), following the French Treebank conventions (http://www.llf.cnrs.fr/Gens/Abeille/French-Treebank-fr.php)
:- raw (no tokenization, no segmentation in sentences), you can activate MElt's embedded lightweight tokenizer by using the '-t' option.
;Tag set :
:[[CC Tagset]] (Crabbé & Candito, 2008)
> A high-accuracy MEMM POS-tagger for French trained on French Treebank and Lefff.
MEMM : [[Maximum-Entropy Markov Model|http://en.wikipedia.org/wiki/Maximum-entropy_Markov_model]]
<<<
MElt is a Python implementation of the MaxEnt Markov Model part-of-speech tagger
described in:
* P. Denis and B. Sagot. 2009. Coupling an annotated corpus and a
morphosyntactic lexicon for state-of-the-art POS tagging with less
human effort. In Proc. of PACLIC 23, Hong Kong, China.
* P. Denis and B. Sagot. 2010. Exploitation d'une ressource lexicale pour la
construction d'un Etiqueteur morphosyntaxique Etat-de-l'art du français.
In Proc. of TALN 2010, Montreal, Canada.
<<<
> To successfully run MElt, you need to install Numpy along with a fairly recent version of Python - we've tried 2.5 and 2.6.
marquage d'unités/expressions multi-mots (Multi-Words Expression)
Ce wiki rassemble quelques informations sur différents POS taggers travaillant avec le français ainsi que quelques autres POS taggers importants (travailllant au moins avec l'anglais).
Les différents tags servent à indexer les articles :
* fonctionalités : ''<<tag POS-tagger "POS-tagger" "Part-of-Speech tagger">>'', <<tag tag-set "tag-set" "jeux de tags">>, <<tag Morpho Morpho "analyse morphologique">>, <<tag Chunker>>, <<tag Parser>>, <<tag Tokenization>>, <<tag [[Sentence Segmentation]]>>, <<tag [[Named Entity]]>>, ''<<tag MWE "MWE" "MultiWord Expression/Unit">>'', <<tag Coreference>>, ''<<tag Toolkit>>'', ...
* langues : français : ''<<tag lang:FR lang:FR français>>'', anglais: <<tag lang:EN lang:EN anglais>>, autres langues :<<listTags inline "%1" ", " lang:.* AND NOT lang:FR AND NOT lang:EN AND NOT lang:FR? >>
* entrées (préfixe "in:") : //raw text// <<tag in:raw>>, //tokenized// <<tag in:sent>>, <<tag in:tok>>, <<tag in:cols>>, //xml// <<tag in:xml>> (nota: le tag <<tag format>> sert pour les formats d'entrée/sortie spécifiques) (autre entrée : <<listTags inline "%1" ", " in:.* AND NOT ( in:raw OR in:sent OR in:tok OR in:cols OR in:xml)>>)
* algorithmes utilisés (préfixe "algo:") : <<listTags inline "%1" ", " algo:.*>>
* langage de programation (préfixe "src:") : <<listTags inline "%1" ", " src:.*>>
* licence (préfixe "licence:") : <<listTags inline "%1" ", " licence:.*>>
Voici la liste des articles de POS-taggers pour le français : <<matchTags popup "label:POS-tagger & lang:FR" POS-tagger AND lang:FR>> <<matchTags "#[[%0]]<br>^^%6^^" "\n" POS-tagger AND lang:FR>>
Et la liste des autres POS-taggers : <<matchTags popup "label:POS-tagger & !lang:FR" POS-tagger AND NOT lang:FR>> <<matchTags "#[[%0]]<br>^^%6^^" "\n" POS-tagger AND NOT lang:FR>>
Liste de tag sets : (tag <<tag tag-set>>):
* pour le français : <<matchTags popup "label:tag-set & lang:FR" tag-set AND lang:FR>> <<matchTags "#[[%0]]" "\n" tag-set AND lang:FR>>
* pour l'anglais : <<matchTags popup "label:tag-set & lang:EN" tag-set AND lang:EN>> <<matchTags "#[[%0]]" "\n" tag-set AND lang:EN>>
L'article [[POS tagger links]] contients quelques liens de listes ou de comparaisons de POS taggers.
<<tagsTree menu-pages "" 2 4 index prettyname>>
<<tagsTree menu "" 1 4 index prettyname>>
<<matchTags popup "label:POS-taggers pour le français" POS-tagger AND lang:FR>>
<<matchTags inline "[[%0]]" "\n" POS-tagger AND lang:FR>>
<<matchTags popup "label:Other POS-tagger" POS-tagger AND NOT lang:FR>>
<<matchTags inline "[[%0]]" "\n" POS-tagger AND NOT lang:FR>>
<<matchTags popup "label:Other tools" (Morpho OR Chunker OR Parsing OR Tokenization OR Sentence Segmentation OR Named Entity OR Coreference) AND NOT POS-tagger>>
<<matchTags inline "[[%0]]" "\n" (Morpho OR Chunker OR Parsing OR Tokenization OR Sentence Segmentation OR Named Entity OR Coreference OR Toolkit) AND NOT POS-tagger>>
<<tag lang:FR "Tous les outils pour le français" "outils/tagsets pour le français">>
<<tag lang:EN "All tools for English" "outils/tagset pour l'anglais">>
[img[http://mallet.cs.umass.edu/logo3.png]] """MAchine Learning for LanguagE Toolkit"""
http://mallet.cs.umass.edu
Possibilité de //Sequence Tagging// (CRF) : [[Quick Start|http://mallet.cs.umass.edu/sequences.php]] [[Developer's Guide|http://mallet.cs.umass.edu/fst.php]]
<<<
MALLET is a Java-based package for statistical natural language processing, document classification, clustering, topic modeling, information extraction, and other machine learning applications to text.
MALLET includes sophisticated tools for document classification: efficient routines for converting text to "features", a wide variety of algorithms (including Naïve Bayes, Maximum Entropy, and Decision Trees), and code for evaluating classifier performance using several commonly used metrics. [[Quick Start|http://mallet.cs.umass.edu/classification.php]] [[Developer's Guide|http://mallet.cs.umass.edu/classifier-devel.php]]
In addition to classification, MALLET includes tools for sequence tagging for applications such as named-entity extraction from text. Algorithms include Hidden Markov Models, Maximum Entropy Markov Models, and Conditional Random Fields. These methods are implemented in an extensible system for finite state transducers. [[Quick Start|http://mallet.cs.umass.edu/sequences.php]] [[Developer's Guide|http://mallet.cs.umass.edu/fst.php]]
Topic models are useful for analyzing large collections of unlabeled text. The MALLET topic modeling toolkit contains efficient, sampling-based implementations of Latent Dirichlet Allocation, Pachinko Allocation, and Hierarchical LDA. [[Quick Start|http://mallet.cs.umass.edu/topics.php]]
Many of the algorithms in MALLET depend on numerical optimization. MALLET includes an efficient implementation of Limited Memory BFGS, among many other optimization methods. [[Developer's Guide|http://mallet.cs.umass.edu/optimization.php]]
In addition to sophisticated Machine Learning applications, MALLET includes routines for transforming text documents into numerical representations that can then be processed efficiently. This process is implemented through a flexible system of "pipes", which handle distinct tasks such as tokenizing strings, removing stopwords, and converting sequences into count vectors. [[Quick Start|http://mallet.cs.umass.edu/import.php]] [[Developer's Guide|http://mallet.cs.umass.edu/import-devel.php]]
An add-on package to MALLET, called GRMM, contains support for inference in general graphical models, and training of CRFs with arbitrary graphical structure. [[About GRMM|http://mallet.cs.umass.edu/grmm/index.php]]
<<<
http://w3.msi.vxu.se/~nivre/research/MaltXML.html
<<<
Malt-TAB is a text-based representation, which is mainly used by MaltParser. Malt-TAB contains a subset of the features in Malt-XML, and attributes are implicitly defined by their position. Each word is represented on one line, with attribute values being separated by tabs. The required order of attributes is as follows:
{{{form (required) < postag (required) < head (optional) < deprel (optional)}}}
Although ''head'' and ''deprel'' are optional, they must either both be included or both be omitted. (Normally, all four columns are present in the input when training the parser and in the output when parsing, while only ''form'' and ''postag'' are present in the input when parsing.) Please note also that the ''id'' attribute is not represented explicitly at all. Words in a sentence are separated by one newline; sentences are separated by one additional newline. A dependency tree for the Swedish sentence "Genom skattereformen införs individuell beskattning (särbeskattning) av arbetsinkomster." can be represented as follows:
|noborder|k
|Genom |pp |3 |ADV |
|skattereformen |nn.utr.sin.def.nom |1 |PR |
|införs |vb.prs.sfo | 0 |ROOT |
|individuell |jj.pos.utr.sin.ind.nom |5 |ATT |
|beskattning |nn.utr.sin.ind.nom |3 |SUB |
|( |pad |5 |IP |
|särbeskattning |nn.utr.sin.ind.nom |5 |APP |
|) |pad | 5 |IP |
|av |pp |5 |ATT |
|arbetsinkomster |nn.utr.plu.ind.nom |9 |PR |
|. |mad |3 |IP |
<<<
http://www.maltparser.org/
;Output/Trainning data :
:- [[CoNLL data format]] 1 token/line, sentence separed by blank line, 10 columns
:- [[Malt-TAB]] 1 token/line, sentence separed by blank line, 4 columns
;Input :
:- the first six columns of the [[CoNLL data format]]
:- [[Malt-TAB]] 1 token/line, 2 columns
<<<
MaltParser can be characterized as a data-driven parser-generator. While a traditional parser-generator constructs a parser given a grammar, a data-driven parser-generator constructs a parser given a treebank. MaltParser is an implementation of inductive dependency parsing, where the syntactic analysis of a sentence amounts to the derivation of a dependency structure, and where inductive machine learning is used to guide the parser at nondeterministic choice points (Nivre, 2006). The parsing methodology is based on three essential components:
# Deterministic parsing algorithms for building labeled dependency graphs (Kudo and Matsumoto,2002; Yamada and Matsumoto, 2003; Nivre,2003)
# History-based models for predicting the next parser action at nondeterministic choice points (Black et al., 1992; Magerman, 1995; Ratnaparkhi, 1997; Collins, 1999)
# Discriminative learning to map histories to parser actions (Kudo and Matsumoto, 2002; Yamada and Matsumoto, 2003; Nivre et al., 2004; Hall et al., 2006)
<<<
<<<
MaltParser implements nine deterministic parsing algorithms:
* Nivre arc-eager
* Nivre arc-standard
* Covington non-projective
* Covington projective
* Stack projective
* Stack swap-eager
* Stack swap-lazy
* Planar (implemented by Carlos Gómez-Rodríguez)
* 2-planar (implemented by Carlos Gómez-Rodríguez)
MaltParser allows users to define feature models of arbitrary complexity.
MaltParser currently includes two machine learning packages (thanks to Sofia Cassel for her work on LIBLINEAR):
* LIBSVM - A Library for Support Vector Machines (Chang, 2001).
* LIBLINEAR -- A Library for Large Linear Classification (Fan et al., 2008).
MaltParser can also be turned into a phrase structure parser that recovers both continuous and discontinuous phrases with both phrase labels and grammatical functions (Hall and Nivre, 2008a; Hall and Nivre, 2008b).
<<<
/***
|Name|MatchTagsPlugin|
|Source|http://www.TiddlyTools.com/#MatchTagsPlugin|
|Documentation|http://www.TiddlyTools.com/#MatchTagsPluginInfo|
|Version|2.0.6|
|Author|Eric Shulman|
|License|http://www.TiddlyTools.com/#LegalStatements|
|~CoreVersion|2.1|
|Type|plugin|
|Description|'tag matching' with full boolean expressions (AND, OR, NOT, and nested parentheses)|
!!!!!Documentation
> see [[MatchTagsPluginInfo]]
!!!!!Revisions
<<<
2011.10.28 2.0.6 added .matchTags CSS class to popups to enable custom styling via StyleSheet
2011.01.23 2.0.5 fix core tweak for TW262+: adjust code in config.filters['tag'] instead of filterTiddlers()
2010.08.11 2.0.4 in getMatchingTiddlers(), fixed sorting for descending order (e.g, "-created")
| please see [[MatchTagsPluginInfo]] for additional revision details |
2008.02.28 1.0.0 initial release
<<<
!!!!!Code
***/
//{{{
version.extensions.MatchTagsPlugin= {major: 2, minor: 0, revision: 6, date: new Date(2011,10,28)};
// store.getMatchingTiddlers() processes boolean expressions for tag matching
// sortfield (optional) sets sort order for tiddlers - default=title
// tiddlers (optional) use alternative set of tiddlers (instead of current store)
TiddlyWiki.prototype.getMatchingTiddlers = function(tagexpr,sortfield,tiddlers) {
var debug=config.options.chkDebug; // abbreviation
var cmm=config.macros.matchTags; // abbreviation
var r=[]; // results are an array of tiddlers
var tids=tiddlers||store.getTiddlers();
if (tids && sortfield) tids=store.sortTiddlers(tids,sortfield);
if (debug) displayMessage(cmm.msg1.format([tids.length]));
// try simple lookup to quickly find single tags or tags that
// contain boolean operators as literals, e.g. "foo and bar"
for (var t=0; t<tids.length; t++)
if (tids[t].isTagged(tagexpr)) r.pushUnique(tids[t]);
if (r.length) {
if (debug) displayMessage(cmm.msg4.format([r.length,tagexpr]));
return r;
}
// convert expression into javascript code with regexp tests,
// so that "tag1 AND ( tag2 OR NOT tag3 )" becomes
// "/\~tag1\~/.test(...) && ( /\~tag2\~/.test(...) || ! /\~tag3\~/.test(...) )"
// normalize whitespace, tokenize operators, delimit with "~"
var c=tagexpr.trim(); // remove leading/trailing spaces
c = c.replace(/\s+/ig," "); // reduce multiple spaces to single spaces
c = c.replace(/\(\s?/ig,"~(~"); // open parens
c = c.replace(/\s?\)/ig,"~)~"); // close parens
c = c.replace(/(\s|~)?&&(\s|~)?/ig,"~&&~"); // &&
c = c.replace(/(\s|~)AND(\s|~)/ig,"~&&~"); // AND
c = c.replace(/(\s|~)?\|\|(\s|~)?/ig,"~||~"); // ||
c = c.replace(/(\s|~)OR(\s|~)/ig,"~||~"); // OR
c = c.replace(/(\s|~)?!(\s|~)?/ig,"~!~"); // !
c = c.replace(/(^|~|\s)NOT(\s|~)/ig,"~!~"); // NOT
c = c.replace(/(^|~|\s)NOT~\(/ig,"~!~("); // NOT(
// change tag terms to regexp tests
var terms=c.split("~"); for (var i=0; i<terms.length; i++) { var t=terms[i];
if (/(&&)|(\|\|)|[!\(\)]/.test(t) || t=="") continue; // skip operators/parens/spaces
if (t==config.macros.matchTags.untaggedKeyword)
terms[i]="tiddlertags=='~~'"; // 'untagged' tiddlers
else
terms[i]="/\\~"+t+"\\~/.test(tiddlertags)";
}
c=terms.join(" ");
if (debug) { displayMessage(cmm.msg2.format([tagexpr])); displayMessage(cmm.msg3.format([c])); }
// scan tiddlers for matches
for (var t=0; t<tids.length; t++) {
// assemble tags from tiddler into string "~tag1~tag2~tag3~"
var tiddlertags = "~"+tids[t].tags.join("~")+"~";
try { if(eval(c)) r.push(tids[t]); } // test tags
catch(e) { // error in test
displayMessage(cmm.msg2.format([tagexpr]));
displayMessage(cmm.msg3.format([c]));
displayMessage(e.toString());
break; // skip remaining tiddlers
}
}
if (debug) displayMessage(cmm.msg4.format([r.length,tagexpr]));
return r;
}
//}}}
//{{{
config.macros.matchTags = {
msg1: "scanning %0 input tiddlers",
msg2: "looking for '%0'",
msg3: "using expression: '%0'",
msg4: "found %0 tiddlers matching '%1'",
noMatch: "no matching tiddlers",
untaggedKeyword: "-",
untaggedLabel: "no tags",
untaggedPrompt: "show tiddlers with no tags",
defTiddler: "MatchingTiddlers",
defTags: "",
defFormat: "[[%0]]",
defSeparator: "\n",
reportHeading: "Found %0 tiddlers tagged with: '{{{%1}}}'\n----\n",
handler: function(place,macroName,params,wikifier,paramString,tiddler) {
var mode=params[0]?params[0].toLowerCase():'';
if (mode=="inline")
params.shift();
if (mode=="report" || mode=="panel") {
params.shift();
var target=params.shift()||this.defTiddler;
}
if (mode=="popup") {
params.shift();
if (params[0]&¶ms[0].substr(0,6)=="label:") var label=params.shift().substr(6);
if (params[0]&¶ms[0].substr(0,7)=="prompt:") var prompt=params.shift().substr(7);
} else {
var fmt=(params.shift()||this.defFormat).unescapeLineBreaks();
var sep=(params.shift()||this.defSeparator).unescapeLineBreaks();
}
var sortBy="+title";
if (params[0]&¶ms[0].substr(0,5)=="sort:") sortBy=params.shift().substr(5);
var expr = params.join(" ");
if (mode!="panel" && (!expr||!expr.trim().length)) return;
if (expr==this.untaggedKeyword)
{ var label=this.untaggedLabel; var prompt=this.untaggedPrompt };
switch (mode) {
case "popup": this.createPopup(place,label,expr,prompt,sortBy); break;
case "panel": this.createPanel(place,expr,fmt,sep,sortBy,target); break;
case "report": this.createReport(target,this.defTags,expr,fmt,sep,sortBy); break;
case "inline": default: this.createInline(place,expr,fmt,sep,sortBy); break;
}
},
formatList: function(tids,fmt,sep) {
var out=[];
for (var i=0; i<tids.length; i++) { var t=tids[i];
var title=t.title;
var who=t.modifier;
var when=t.modified.toLocaleString();
var text=t.text;
var first=t.text.split("\n")[0];
var desc=store.getTiddlerSlice(t.title,"description");
desc=desc||store.getTiddlerSlice(t.title,"Description");
desc=desc||store.getTiddlerText(t.title+"##description");
desc=desc||store.getTiddlerText(t.title+"##Description");
var tags=t.tags.length?'[['+t.tags.join(']] [[')+']]':'';
out.push(fmt.format([title,who,when,text,first,desc,tags]));
}
return out.join(sep);
},
createInline: function(place,expr,fmt,sep,sortBy) {
wikify(this.formatList(store.sortTiddlers(store.getMatchingTiddlers(expr),sortBy),fmt,sep),place);
},
createPopup: function(place,label,expr,prompt,sortBy) {
var btn=createTiddlyButton(place,
(label||expr).format([expr]),
(prompt||config.views.wikified.tag.tooltip).format([expr]),
function(ev){ return config.macros.matchTags.showPopup(this,ev||window.event); });
btn.setAttribute("sortBy",sortBy);
btn.setAttribute("expr",expr);
},
showPopup: function(here,ev) {
var p=Popup.create(here,null,"matchTags popup"); if (!p) return false;
var tids=store.getMatchingTiddlers(here.getAttribute("expr"));
store.sortTiddlers(tids,here.getAttribute("sortBy"));
var list=[]; for (var t=0; t<tids.length; t++) list.push(tids[t].title);
if (!list.length) createTiddlyText(p,this.noMatch);
else {
var b=createTiddlyButton(createTiddlyElement(p,"li"),
config.views.wikified.tag.openAllText,
config.views.wikified.tag.openAllTooltip,
function() {
var list=this.getAttribute("list").readBracketedList();
story.displayTiddlers(null,tids);
});
b.setAttribute("list","[["+list.join("]] [[")+"]]");
createTiddlyElement(p,"hr");
}
var out=this.formatList(tids," [[%0]] ","\n"); wikify(out,p);
Popup.show();
ev.cancelBubble=true;
if(ev.stopPropagation) ev.stopPropagation();
return false;
},
createReport: function(target,tags,expr,fmt,sep,sortBy) {
var tids=store.sortTiddlers(store.getMatchingTiddlers(expr),sortBy);
if (!tids.length) { displayMessage('no matches for: '+expr); return false; }
var msg=config.messages.overwriteWarning.format([target]);
if (store.tiddlerExists(target) && !confirm(msg)) return false;
var out=this.reportHeading.format([tids.length,expr])
out+=this.formatList(tids,fmt,sep);
store.saveTiddler(target,target,out,config.options.txtUserName,new Date(),tags,{});
story.closeTiddler(target); story.displayTiddler(null,target);
},
createPanel: function(place,expr,fmt,sep,sortBy,tid) {
var s=createTiddlyElement(place,"span"); s.innerHTML=store.getTiddlerText("MatchTagsPlugin##html");
var f=s.getElementsByTagName("form")[0];
f.expr.value=expr; f.fmt.value=fmt; f.sep.value=sep.escapeLineBreaks();
f.tid.value=tid; f.tags.value=this.defTags;
}
};
//}}}
/***
//{{{
!html
<form style='display:inline;white-space:nowrap'>
<input type='text' name='expr' style='width:50%' title='tag expression'><!--
--><input type='text' name='fmt' style='width:10%' title='list item format'><!--
--><input type='text' name='sep' style='width:5%' title='list item separator'><!--
--><input type='text' name='tid' style='width:12%' title='target tiddler title'><!--
--><input type='text' name='tags' style='width:10%' title='target tiddler tags'><!--
--><input type='button' name='go' style='width:8%' value='go' onclick="
var expr=this.form.expr.value;
if (!expr.length) { alert('Enter a boolean tag expression'); return false; }
var fmt=this.form.fmt.value;
if (!fmt.length) { alert('Enter the list item output format'); return false; }
var sep=this.form.sep.value.unescapeLineBreaks();
var tid=this.form.tid.value;
if (!tid.length) { alert('Enter a target tiddler title'); return false; }
var tags=this.form.tags.value;
config.macros.matchTags.createReport(tid,tags,expr,fmt,sep,'title');
return false;">
</form>
!end
//}}}
***/
//{{{
// SHADOW TIDDLER for displaying default panel input form
config.shadowTiddlers.MatchTags="<<matchTags panel>>";
//}}}
//{{{
// TWEAK core filterTiddlers() or config.filters['tag'] (in TW262+)
// to use getMatchingTiddlers instead getTaggedTiddlers
// for enhanced boolean matching in [tag[...]] syntax
var TW262=config.filters && config.filters['tag']; // detect TW262+
var fname=TW262?"config.filters['tag']":"TiddlyWiki.prototype.filterTiddlers";
var code=eval(fname).toString().replace(/getTaggedTiddlers/g,'getMatchingTiddlers');
eval(fname+'='+code);
//}}}
//{{{
// REDEFINE core handler for enhanced boolean matching in tag:"..." paramifier
// use filterTiddlers() instead of getTaggedTiddlers() to get list of tiddlers.
config.paramifiers.tag = {
onstart: function(v) {
var tagged = store.filterTiddlers("[tag["+v+"]]");
story.displayTiddlers(null,tagged,null,false,null);
}
};
//}}}
/***
|Name|MatchTagsPluginInfo|
|Source|http://www.TiddlyTools.com/#MatchTagsPlugin|
|Documentation|http://www.TiddlyTools.com/#MatchTagsPluginInfo|
|Version|2.0.6|
|Author|Eric Shulman|
|License|http://www.TiddlyTools.com/#LegalStatements|
|~CoreVersion|2.1|
|Type|documentation|
|Description|documentation for MatchTagsPlugin|
!!!!!Usage
<<<
This plugin extends the {{{[tag[tagname]]}}} macro parameter syntax used by the TiddlyWiki core {{{<<list>>}}} macro so that, instead of a simple tagname value, you can specify a complex combination of tagname values using a //boolean expression// containing AND, OR, and NOT operators, enclosed in nested parentheses if needed.
{{{
<<list filter "[tag[expression]]">>
}}}
In addition, the plugin defines a new macro, {{{<<matchTags ...>>}}} that can be used instead of the core {{{<<list>>}}} macro to output a list of matching tiddlers //using a custom 'item format' and 'separator'//. You can also use this macro to create a command link that displays the matching tiddlers within a popup list, similar to the standard {{{<<tag tagName>>}}} macro, but matching a combination of tag values rather than a single tag value.
{{{
<<matchTags inline "format" "separator" sort:fieldname tag expression>>
<<matchTags popup "label:..." "prompt:..." sort:fieldname tag expression>>
<<matchTags report TiddlerName "format" "separator" sort:fieldname tag expression>>
<<matchTags panel Tiddlername "format" "separator" sort:fieldname tag expression>>
}}}
where:
* ''inline'', ''report'', ''panel'', and ''popup''<br>are keywords that indicate the type of output that the macro should produce:
** ''inline'' //(default)// - displays a list of matching tiddlers embedded directly in tiddler content
** ''popup'' - embeds a command button that, when clicked, lists matching tiddlers in a ~TiddlyWiki popup display
** ''report'' - generates a list of matching tiddler in a separate [[MatchingTiddlers]] report tiddler
** ''panel'' - displays an interactive form for generating a [[MatchingTiddlers]] report
* ''format''<br>defines the wiki-syntax for rendering list items. The following //substitution markers// can be used to insert tiddler-specific information for each matched tiddler:
** {{{%0}}} - title
** {{{%1}}} - modifier (author)
** {{{%2}}} - modified (date of last change)
** {{{%3}}} - text (all tiddler content)
** {{{%4}}} - firstline (tiddler content up to the first newline)
** {{{%5}}} - description (tiddler slice or section content named "description" or "Description")
** {{{%6}}} - tags (space-separated, bracketed list)
* ''separator''<br>defines the wiki-syntax to use //between// each matching title (e.g., ", " creates a comma-separated list, while "\n" displays one tiddler per line).
* ''sort:fieldname'' (optional)<br>specifies the sort order for the resulting list of tiddlers. You can specify any tiddler field name (standard or custom-defined). Standard tiddler fieldnames include: //title, created, modified, modifier//. If not specified, tiddlers are sorted by title. You can prefix the fieldname with "+" or "-" to indicate ascending or descending order, respectively.
* ''tag expression''<br>the remaining parameter(s) are joined together to define the boolean expression to be matched.
When using the ''popup'' option, there are two additional (and optional) parameters you can specify:
* ''"label:..."''(optional)<br> indicates the text for the popup command link. The default is to display the specified tag expression itself.
* ''"prompt:..."'' (optional)<br>indicates the mouseover 'tooltip' for the popup command link.
* note: you can apply custom CSS styles (e.g., font size) to the popup by adding a rule for ".matchTags .popup" to your [[StyleSheet]].
When using the ''report'' or ''panel'' option, an additional parameter may be provided:
* ''~TiddlerName''<br>specifies the target tiddler into which the output will be generated (default: [[MatchingTiddlers]])
Notes:
*A tag expression can use any combination of text operators: ''AND'', ''OR'', ''NOT'' (or their equivalent javascript operators: ''&&'', ''||'', ''!''), contained in nested parentheses as needed.
*Operators should be delimited by spaces or parentheses.
*Before matching, leading/trailing spaces are automatically trimmed and multiple spaces are reduced to single spaces.
*Tag values containing embedded spaces do //not// have to be enclosed in {{{[[...]]}}}.
*Tag values that contain boolean operators as ''literal text'' (e.g., {{{"foo and bar"}}} or {{{"foo && bar"}}} cannot be used within a compound boolean expression, but //can// be matched if specified by themselves, without any other tag values or operators.
*To match tiddlers that are untagged, use "-" as a special tag value within the expression.
*You can match "wildcard" tags by using //regular expression// (i.e., "text pattern") syntax within a tag value, e.g. {{{[Tt]agvalue.*}}}
<<<
!!!!!Examples:
<<<
display a popup list:
{{{
<<matchTags popup sample OR (settings AND systemConfig)>>
}}}
><<matchTags popup sample OR (settings AND systemConfig)>>
display a popup list with custom label:
{{{
<<matchTags popup "label:samples and settings" sample OR (settings AND systemConfig)>>
}}}
><<matchTags popup "label:samples and settings" sample OR (settings AND systemConfig)>>
display a popup list of untagged tiddlers:
{{{
<<matchTags popup ->>
}}}
><<matchTags popup ->>
generate a report using interactive form control panel
{{{
<<matchTags panel "MatchingTiddlers" "[[%0]]" "\n" sample OR (settings AND systemConfig)>>
}}}
>{{smallform{<<matchTags panel "MatchingTiddlers" "[[%0]]" "\n" sample OR (settings AND systemConfig)>>}}}
comma-separated list:
{{{
<<matchTags "[[%0]]" ", " sample OR (settings AND systemConfig)>>
}}}
><<matchTags "[[%0]]" ", " sample OR (settings AND systemConfig)>>
numbered list (sorted by modification date, most recent first):
{{{
<<matchTags "#[[%0]] (%2)<br>^^%5^^" "\n" sort:-modified sample OR (settings AND systemConfig)>>
}}}
><<matchTags "#[[%0]] (%2)<br>^^%5^^" "\n" sort:-modified sample OR (settings AND systemConfig)>>
bullet-item list (using the TiddlyWiki core {{{<<list filter ...>>}}} macro):
//(Note: when using the core {{{<<list>>}}} macro, you should always enclose the entire tag filter parameter within quotes)//
{{{
<<list filter "[tag[sample OR (settings AND systemConfig)]]">>
}}}
><<list filter "[tag[sample OR (settings AND systemConfig)]]">>
<<<
!!!!!Revisions
<<<
2011.10.28 2.0.6 added .matchTags CSS class to popups to enable custom styling via StyleSheet
2011.01.22 2.0.5 fix core tweak for TW262+: adjust code in config.filters['tag'] instead of filterTiddlers()
2010.08.11 2.0.4 in getMatchingTiddlers(), fixed sorting for descending order (e.g, "-created")
2010.03.02 2.0.3 added %6 format (tags)
2010.03.01 2.0.2 in formatList(), don't automatically put '[[' and ']]' around title (%0) in formatted output
2009.08.29 2.0.1 added support for {{{config.macros.matchTags.defTags}}} to auto-tag [[MatchingTiddlers]] output
2008.09.04 2.0.0 added "report" and "panel" options to generate formatted results and store in a tiddler. Also, added config.macros.matchTags.formatList(place,fmt,sep) API to return formatted output for use with other plugins/scripts
2008.09.01 1.9.2 fixed return value from popup button handler so IE doesn't attempt to leave the page
2008.08.31 1.9.1 improved expression conversion handling to permit use of regular expressions for "wildcard" matching within tag values
2008.06.12 1.9.0 added support for formatted output of: title, who, when, text, firstline, description (slice or section)
2008.06.05 1.8.0 in getMatchingTiddlers(), added optional sortfield and tiddlers params to support use of alternative set of tiddlers instead of using current store content (provides filtering support for ImportTiddlersPlugin)
2008.06.04 1.7.1 in getMatchingTiddlers(), reworked conversion of expression for more robust parsing of whitespace, parentheses and javascript operators and allow use of "-" (untagged) //within// expressions
2008.05.19 1.7.0 in getMatchingTiddlers(), use reverseLookup() instead of forEachTiddler() to permit access to tiddlers included via [[IncludePlugin|http://tiddlywiki.abego-software.de/#IncludePlugin]]
2008.05.17 1.6.0 in getMatchingTiddlers(), rewrote expression conversion to handle tags with spaces tag values that are substrings of other tag values.
2008.05.16 1.5.0 added special case using "-" to find UNTAGGED tiddlers
2008.05.15 1.4.0 added "popup" output option
2008.05.14 1.3.4 instead of hijacking getTaggedTiddlers(), added tweak of filterTiddlers() prototype to replace getTaggedTiddlers() with getMatchingTiddler() so that core use of getTaggedTiddlers() does not perform boolean processing of tiddler titles such as [[To Be or not To Be]]. Also, improved "filter error" messages in getMatchingTiddlers() to report tag expression in addition to actual eval error.
2008.04.25 1.3.3 in getTaggedTiddlers(), fixed handling for "not" embedded within a tag
2008.04.21 1.3.2 in getTaggedTiddlers(), fixed handling for initial "NOT" and "NOT(expr)" syntax
2008.04.20 1.3.1 in getTaggedTiddlers(), corrected check for boolean expression to avoid excess processing of tags containing spaces. Also, improved handling for non-existing tags that contain text of existing tags
2008.04.19 1.3.0 in filterTiddlers(), use getTaggedTiddlers() instead of matchTags(), and then hijack getTaggedTiddlers() to add matchTags() handling
2008.04.19 [*.*.*] plugin size reduction: moved documentation to [[MatchTagsPluginInfo]]
2008.03.25 1.2.0 added optional "sort:fieldname" parameter
2008.03.20 1.1.2 in handler(), replace 'encodeTiddlyLink' with explicit [[...]] brackets to ensure that one-word tiddler titles are properly rendered as TiddlyLinks
2008.02.29 1.1.1 in matchTags(), added handling to skip remaining tiddlers if expression has an error
2008.02.29 1.1.0 refactored to define store.matchTags() and extend store.filterTiddlers()
2008.02.28 1.0.0 initial release
<<<
https://sites.google.com/site/morfetteweb/
;Input : (''UTF-8'')
:tokenized text, 1 token/line, blank line separate sentences
;Output/Training data :
:one token per line, with three columns separated by __spaces or tabs__. The columns contain word form, lemma and morphological tag respectively. Sentences are separated by an empty line. Text should be encoded in UTF-8.
:{{{FORM LEMMA MORPHO-TAG}}}
;Lexicon format :
:one entry per line, fields in each entry separated by whitespace in the UTF-8 encoding.
:{{{FORM LEMMA1 MORPHO-TAG1 LEMMA2 MORPHO-TAG2 ...}}}
;Tag sets:
:ES: Model trained on approx. 168000 tokens from the __Spanish AnCora treebank__ (Marti et al 2007).Lexicon features extracted from the __Spanish Resource Grammar project__, contains over 556,000 word forms
:FR: Model trained on approx. 277000 tokens from the __French Treebank__ (Abeillé et al, 2003). Lexicon features extracted from the __LeFFF lexicon__ (Sagot et al, 2006). The version we used contains over 225000 word forms
> Morfette is a tool for supervised learning of inflectional morphology. Given a corpus of sentences annotated with lemmas and morphological labels, and optionally a lexicon, morfette learns how to morphologically analyse new sentences.
> In the learning stage Morfette fits two separate logistic regression models: one for morphological tagging and one for lemmatization. The predictions of the models are combined dynamically and produce a globally plausible sequence of morphological-tag - lemma pairs for a sentence.
> In Morfette lemmatization is cast as a classification task where a a lemmatization class corresponds to the specification of the edit operations which are needed to transform the inflected word form into the corresponding lemma.
> The basic approach is described in (Chrupala et al 2008 and Chrupala 2008). The current version of Morfette uses an averaged perceptron to fit the models, rather than Maximum Entropy training. The lemmatization classes are Edit-Tree-based as described in (Chrupala 2008).
étiquetage morphosyntaxique (plus fin que les POS, et souvent non-désambiguïsé)
[img[http://aune.lpl.univ-aix.fr/projects/multext/MUL7.multext.gif]]
http://aune.lpl.univ-aix.fr/projects/multext/MUL7.html
http://www.issco.unige.ch/en/research/projects/MULTEXT.html
<<<
Morpho-lexical tools
* [[MtLex|http://aune.lpl.univ-aix.fr/projects/multext/MtLex/]] -- Multext lexical access tools
* [[MtTag|ftp://issco-ftp.unige.ch/pub/multext/]] - Multext POS disambiguator and related utilities
* [[MtMorph|ftp://issco-ftp.unige.ch/pub/multext/]] - Multext morphological tools
<<<
<<<
Text segmentation tools
* [[MtSeg|http://aune.lpl.univ-aix.fr/projects/multext/MtSeg/]] - Text segmenter
<<<
''MtTag'' (POS disambiguator) ne semble plus disponible, l'ISSCO de l'Université de Genève propose un succésseur : [[TATOO]]
* Association for Computational Linguistics wiki
** Part-of-speech tagging : http://aclweb.org/aclwiki/index.php?title=Part-of-speech_tagging
** POS Tagging (State of the art) : http://aclweb.org/aclwiki/index.php?title=POS_Tagging_%28State_of_the_art%29
* Stanford list of NLP resources : http://www-nlp.stanford.edu/links/statnlp.html#Taggers
* TAUS Directory of tools, section //POS and lemma annotators (taggers)// : https://directories.taus.net/pos-and-lemma-annotators/211-pos-and-lemma-annotators-taggers
<!--{{{-->
<div class='header' role='banner' macro='gradient vert [[ColorPalette::PrimaryLight]] [[ColorPalette::PrimaryMid]]'>
<div class='headerShadow'>
<span class='siteTitle' refresh='content' tiddler='SiteTitle'></span>
<span class='siteSubtitle' refresh='content' tiddler='SiteSubtitle'></span>
</div>
<div class='headerForeground'>
<span class='siteTitle' refresh='content' tiddler='SiteTitle'></span>
<span class='siteSubtitle' refresh='content' tiddler='SiteSubtitle'></span>
</div>
<span class='headerMenu' refresh='content' tiddler='HeaderMenu'></span>
</div>
<div id='mainMenu' role='navigation' refresh='content' tiddler='MainMenu'></div>
<div id='sidebar'>
<div id='sidebarOptions' role='navigation' refresh='content' tiddler='SideBarOptions'></div>
<div id='sidebarTabs' role='complementary' refresh='content' force='true' tiddler='SideBarTabs'></div>
</div>
<div id='displayArea' role='main'>
<div id='messageArea'></div>
<div id='tiddlerDisplay'></div>
</div>
<!--}}}-->
analyse syntaxique profonde
/***
|''Name:''|PasswordOptionPlugin|
|''Description:''|Extends TiddlyWiki options with non encrypted password option.|
|''Version:''|1.0.2|
|''Date:''|Apr 19, 2007|
|''Source:''|http://tiddlywiki.bidix.info/#PasswordOptionPlugin|
|''Author:''|BidiX (BidiX (at) bidix (dot) info)|
|''License:''|[[BSD open source license|http://tiddlywiki.bidix.info/#%5B%5BBSD%20open%20source%20license%5D%5D ]]|
|''~CoreVersion:''|2.2.0 (Beta 5)|
***/
//{{{
version.extensions.PasswordOptionPlugin = {
major: 1, minor: 0, revision: 2,
date: new Date("Apr 19, 2007"),
source: 'http://tiddlywiki.bidix.info/#PasswordOptionPlugin',
author: 'BidiX (BidiX (at) bidix (dot) info',
license: '[[BSD open source license|http://tiddlywiki.bidix.info/#%5B%5BBSD%20open%20source%20license%5D%5D]]',
coreVersion: '2.2.0 (Beta 5)'
};
config.macros.option.passwordCheckboxLabel = "Save this password on this computer";
config.macros.option.passwordInputType = "password"; // password | text
setStylesheet(".pasOptionInput {width: 11em;}\n","passwordInputTypeStyle");
merge(config.macros.option.types, {
'pas': {
elementType: "input",
valueField: "value",
eventName: "onkeyup",
className: "pasOptionInput",
typeValue: config.macros.option.passwordInputType,
create: function(place,type,opt,className,desc) {
// password field
config.macros.option.genericCreate(place,'pas',opt,className,desc);
// checkbox linked with this password "save this password on this computer"
config.macros.option.genericCreate(place,'chk','chk'+opt,className,desc);
// text savePasswordCheckboxLabel
place.appendChild(document.createTextNode(config.macros.option.passwordCheckboxLabel));
},
onChange: config.macros.option.genericOnChange
}
});
merge(config.optionHandlers['chk'], {
get: function(name) {
// is there an option linked with this chk ?
var opt = name.substr(3);
if (config.options[opt])
saveOptionCookie(opt);
return config.options[name] ? "true" : "false";
}
});
merge(config.optionHandlers, {
'pas': {
get: function(name) {
if (config.options["chk"+name]) {
return encodeCookie(config.options[name].toString());
} else {
return "";
}
},
set: function(name,value) {config.options[name] = decodeCookie(value);}
}
});
// need to reload options to load passwordOptions
loadOptionsCookie();
/*
if (!config.options['pasPassword'])
config.options['pasPassword'] = '';
merge(config.optionsDesc,{
pasPassword: "Test password"
});
*/
//}}}
Tagset used by : <<list filter [tag[tags:Penn]]>>
Aoife Cahill's list : http://www.computing.dcu.ie/~acahill/tagset.html
|CC |Coordinating conjunction <br>e.g. and,but,or...|
|CD |Cardinal Number |
|DT |Determiner |
|EX |Existential there |
|FW |Foreign Word |
|IN |Preposision or subordinating conjunction |
|JJ |Adjective |
|JJR |Adjective, comparative |
|JJS |Adjective, superlative |
|LS |List Item Marker |
|MD |Modal <br>e.g. can, could, might, may...|
|NN |Noun, singular or mass |
|NNP |Proper Noun, singular |
|NNPS |Proper Noun, plural |
|NNS |Noun, plural |
|PDT |Predeterminer <br>e.g. all, both ... when they precede an article|
|POS |Possessive Ending <br>e.g. Nouns ending in 's|
|PRP |Personal Pronoun <br>e.g. I, me, you, he...|
|PRP$ |Possessive Pronoun <br>e.g. my, your, mine, yours...|
|RB |Adverb <br>Most words that end in -ly as well as degree words like quite, too and very|
|RBR |Adverb, comparative <br>Adverbs with the comparative ending -er, with a strictly comparative meaning.|
|RBS |Adverb, superlative |
|RP |Particle |
|SYM |Symbol <br>Should be used for mathematical, scientific or technical symbols|
|TO |to |
|UH |Interjection <br>e.g. uh, well, yes, my...|
|VB |Verb, base form <br>subsumes imperatives, infinitives and subjunctives|
|VBD |Verb, past tense <br>includes the conditional form of the verb to be|
|VBG |Verb, gerund or persent participle |
|VBN |Verb, past participle |
|VBP |Verb, non-3rd person singular present |
|VBZ |Verb, 3rd person singular present |
|WDT |Wh-determiner <br>e.g. which, and that when it is used as a relative pronoun|
|WP |Wh-pronoun <br>e.g. what, who, whom...|
|WP$ |Possessive wh-pronoun <br>e.g.|
|WRB |Wh-adverb <br>e.g. how, where why|
|Punctuation Tags|h
|#|
|$|
|"|
|(|
|)|
|,|
|.|
|:|
|`` |
Autres liens :
* AMALGAM page : http://www.comp.leeds.ac.uk/amalgam/tagsets/upenn.html
* [[1993 Computational Linguistics article in PDF|http://acl.ldc.upenn.edu/J/J93/J93-2004.pdf]]
* TreeTagger : http://www.ims.uni-stuttgart.de/projekte/corplex/TreeTagger/Penn-Treebank-Tagset.pdf
SEM : Segmenteur-Étiqueteur Markovien
[img[http://www.lattice.cnrs.fr/plugins/kitcnrs/images/lattice.jpg]]http://www.lattice.cnrs.fr/sites/itellier/SEM.html
;Input :
:- tokenized text : 1 token/line
:- raw text (segmentation dite " maximale ")
;Output :
:le texte est réécrit sous forme de phrase avec son étiquetage POS et son parenthésage en chunks
: {{{(Tout/DET nouvel/ADJ organisme/NC public/ADJ national/ADJ)NP (sera/V implanté/VPP)VN (hors_d'/P Ile-de-France/NPP)PP (./PONCT)O}}}
;Tagset:
:[[SEM Tagset]]
Use [[Wapiti]] a simple and fast discriminative sequence labelling toolkit (http://wapiti.limsi.fr/)
Tagset used by : <<list filter [tag[tags:SEM]]>>
http://www.lattice.cnrs.fr/sites/itellier/SEM.html
''POS'' - L'étiqueteur se base sur le [[jeu d'étiquettes morpho-syntaxiques de (Crabbé et al. 08)|CC Tagset]], auquel on a ajouté une étiquette récupérée de l'application sur le French TreeBank, CL, pour les clitiques dont on ne trouve pas la sous-catégorie, c'est-à-dire les explétifs qui ne sont explicitement considérés ni comme sujets, ni objets, ni réfléchis (typiquement " y " dans " il y a " ou le démonstratif " c' ").
Voici la liste exhaustive des étiquettes utilisées et la catégorie syntaxique à laquelle ils correspondent :
|ADJ |adjectif |
|ADJWH |adjectif interrogatif |
|ADV |adverbe |
|ADVWH |adverbe interrogatif |
|CC |conjonction de coordination |
|CL |pronom clitique |
|CLO |pronom clitique objet |
|CLR |pronom clitique réfléchi |
|CLS |pronom clitique sujet |
|CS |conjonction de subordination |
|DET |déterminant |
|DETWH |déterminant interrogatif |
|ET |mot tiré d'une langue étrangère |
|I |interjection |
|NC |nom commun |
|NPP |nom propre |
|P |préposition |
|P+D |forme contractée préposition et déterminant |
|P+PRO |forme contractée préposition et pronom |
|PONCT |ponctuation |
|PREF |préfixe |
|PRO |pronom |
|PROREL |pronom relatif |
|PROWH |pronom interrogatif |
|V |verbe |
|VIMP |forme verbale à l'impératif |
|VINF |forme verbale à l'infinitif |
|VPP |participe passé |
|VPR |participe présent |
|VS |forme verbale au subjonctif |
''Chunks'' - Quant à la liste des chunks, elle est directement liée à celle des étiquettes POS et comporte donc 6 grands types de groupes (têtes potentielles entre parenthèses) :
|__UNKNOWN__ |chunk de nature non-identifiée (ET) |
|AP |chunk adjectival (ADJ, ADJWH) |
|AdP |chunk adverbial (ADV, ADVWH, I) |
|CONJ |chunk conjonction (CC, CS) |
|NP |chunk nominal (CLO, CLR, CLS, NC, NPP, PRO, PROREL, PROWH) |
|PP |chunk prépositionnel (P, P+D, P+PRO) |
|VN |chunk verbal (V, VIMP, VINF, VPP, VPR, VS) |
[img[http://www.lsi.upc.edu/~nlp/SVMTool/img/logo_talp-upc.jpg]] http://www.lsi.upc.edu/~nlp/SVMTool/
;Input :
:- tokenized text : 1 token/line, the token is expected to be the first column of the line. Lines beginning with ’## ’ are ignored by the tagger
:- multicolumns : 1 token/line, the token is expected to be the first column of the line. (The tag to predict takes the second column in the output.) The rest of the line may contain additional information.
;Output :
:The predicted tag will take the second column in the output. The rest of the line remains unchanged.
;Trainning Data :
:Training data must be in column format, i.e. a token per line corpus in a sentence by sentence fashion. The column separator is the blank space. The token is expected to be the first column of the line. The tag to predict takes the second column in the output. The rest of the line may contain additional information.
<<<
Here you can find information about the SVMTool, an open source generator of sequential taggers. The SVMTool has been developed at TALP Research Center NLP group , in Universitat Politècnica de Catalunya.
The SVMTool is a simple and effective generator of sequential taggers based on Support Vector Machine. We have applied the SVMTool to a number of NLP problems, such as Part-of-speech Tagging and Base Phrase Chunking, for different languages. The proposed SVM-based tagger is robust and flexible for feature modelling (including lexicalization), trains efficiently with almost no parameters to tune, and is able to tag thousands of words per second, which makes it really practical for real NLP applications. Regarding accuracy, the SVM-based tagger achieves a very competitive accuracy of 97.2% for English on the Wall Street Journal corpus, which is comparable to the best taggers reported up to date.
The SVM^^light^^ software implementation of Vapnik's Support Vector Machine [Vapnik, 1995] by Thorsten Joachims has been used to train the models. For further information on it see [[here|http://www.joachims.org/publications/joachims_99a.pdf]] or visit http://svmlight.joachims.org/
Through this web site you will be able to download the SVMTool software. You can also download several models to tag in different languages and models to deal with noisy and ungrammatical texts as those studied in the FAUST project.
<<<
[img[http://www.sygtext.fr/images/logo_syg.gif]] SYGFRAN, analyse morpho-syntaxique, dévellopé par Jacques CHAUCHÉ au [[LIRMM|http://www.lirmm.fr]]
http://www.lirmm.fr/~chauche/SourcesAnalyse/SYGFRAN.html
> L'analyseur SYGFRAN est écrit en SYGMART. Il génère une analyse en constituants d'un énoncé en français.
<<<
Les différents fichiers sources sont accessibles ici (tous sont codés en utf8) :
* [[Le fichier des variables|http://www.lirmm.fr/~chauche/SourcesAnalyse/Variables.gz]]
* [[Le fichier des formats|http://www.lirmm.fr/~chauche/SourcesAnalyse/Formats.gz]]
* [[Le fichier dictionnaire des segments (OPALE)|http://www.lirmm.fr/~chauche/SourcesAnalyse/Dictfr.op.gz]]
* [[Le fichier dictionnaire d'étiquettes (OPALE)|http://www.lirmm.fr/~chauche/SourcesAnalyse/Dictopq.op.gz]]
* [[Le fichier de la grammaire OPALE|http://www.lirmm.fr/~chauche/SourcesAnalyse/Gramop.op.gz]]
* [[Le fichier dictionnaire d'analyse (TELESI)|http://www.lirmm.fr/~chauche/SourcesAnalyse/Dictanfr.tl.gz]]
* [[Le fichier de la grammaire TELESI|http://www.lirmm.fr/~chauche/SourcesAnalyse/Gramanfr.tl.gz]]
<<<
> [[Serveur SYGFRAN en ligne|http://www.lirmm.fr/~chauche/ExempleAnl.html]]
<<search>><<closeAll>><<permaview>><<newTiddler>><<newJournal "DD MMM YYYY" "journal">><<saveChanges>><<tiddler TspotSidebar>>[[(on tiddlyspot)|http://french-postaggers.tiddlyspot.com/index.html]]<<slider chkSliderOptionsPanel OptionsPanel "options »" "Change TiddlyWiki advanced options">>
petite étude de POS taggers pour le français
[img[http://nlp.stanford.edu/img/nlp-logo-navbar.jpg]] http://nlp.stanford.edu/software/tagger.shtml
(@@TODO: check french language@@)
;Input :
:- raw text
:- tokenized text
:- xml (text content inside specific tag(s))
;Output :
:append POS to each word :
: {{{This/DT is/VBZ a/DT sample/NN sentence/NN}}}
;Tag set :
: EN : [[Penn Treebank English POS tag set]]
: FR : [[French TreeBank POS Tags]]
> This software is a Java implementation of the log-linear part-of-speech taggers described in these papers (if citing just one paper, cite the 2003 one):
>
> Kristina Toutanova and Christopher D. Manning. 2000. [[Enriching the Knowledge Sources Used in a Maximum Entropy Part-of-Speech Tagger|http://nlp.stanford.edu/~manning/papers/emnlp2000.pdf]]. In Proceedings of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora (EMNLP/VLC-2000), pp. 63-70.
>
> Kristina Toutanova, Dan Klein, Christopher Manning, and Yoram Singer. 2003. [[Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network|http://nlp.stanford.edu/~manning/papers/tagging.pdf]]. In Proceedings of HLT-NAACL 2003, pp. 252-259.
>
> The tagger was originally written by Kristina Toutanova. Since that time, Dan Klein, Christopher Manning, William Morgan, Anna Rafferty, Michel Galley, and John Bauer have improved its speed, performance, usability, and support for other languages.
> The system requires Java 1.6+ to be installed. Depending on whether you're running 32 or 64 bit Java and the complexity of the tagger model, you'll need somewhere between 60 and 200 MB of memory to run a trained tagger (i.e., you may need to give java an option like java -mx200m). Plenty of memory is needed to train a tagger. It again depends on the complexity of the model but at least 1GB is usually needed, often more.
> Part-of-speech name abbreviations: The English taggers use the Penn Treebank tag set. Here are some links to documentation of the Penn Treebank English POS tag set: [[1993 Computational Linguistics article in PDF|http://acl.ldc.upenn.edu/J/J93/J93-2004.pdf]], [[AMALGAM page|http://www.comp.leeds.ac.uk/amalgam/tagsets/upenn.html]], [[Aoife Cahill's list|http://www.computing.dcu.ie/~acahill/tagset.html]]. See the included README-Models.txt in the models directory for more information about the tagsets for the other languages.
> This software provides a GUI demo, a command-line interface, and an API. Simple scripts are included to invoke the tagger.
.viewer .noborder,
.viewer .noborder tr,
.viewer .noborder th,
.viewer .noborder td {
border: 0;
border-collapse:collapse
}
.headerMenu {
float:right;
}
.headerMenu a {color:[[ColorPalette::Background]];}
.headerMenu .button {border:0px}
TATOO : ISSCO TAgger TOOl
[img[http://www.issco.unige.ch/images/ISSCO-logo90.png]] http://www.issco.unige.ch/en/staff/robert/tatoo/tatoo.html
;Input:
:- a record/field format as given by the Multext segmenter with sentence boundaries marked and with words annotated by lexical look-up (either mtlexax or mmorph look-up)
:- multi-colonnes
''If you want this documentation available offline, copy this tiddler to your TW.''
!Description:
The tagger plugin is a result of combining key features from the dropTags and tagAdder macro's. However, since it departs somewhat from the interface tagAdder users will be familiar with, I'm making this available as a new plugin alongside tagAdder.
Tagger provides a dropdown list of the current tiddler tags, along with the ability to toggle them. Further it can optionally display a list of tags in the dropdown, which can be addded to the tiddler.
*Clicking on ''[x]'' and ''[ ]'' removes and adds the tag respectively.
*Clicking on the tag text displays the tag dropdown for that tag, listing tiddlers tagged with it.
*The ''Create new tag'' lets you quickly type in a new tag not in the list.
*Click on this button to see the dropdown: <<tagger>>
Further note that each tag dropdown has a ''Rename tag'' option. This can be used to quickly rename a tag in the entire TW, also rename it's tiddler if it exists.
//''tagAdder, dropTags and the future''
- tagAdder will no longer will be developed, but will remain available. I encourage all tagAdder users to upgrade to tagger.
- dropTags will still be developed for those users that dont want the 'tag editing' features.//
!Examples & Usage:
*At it's simplest, using tagger is as simple as {{{<<tagger>>}}} <<tagger>>
**This gives a dropdown with the current tiddler tags, followed by all the tags in the TW.
*You can also use a list of specified tags instead of all tags in the TW, by specifying a source tiddler.
**{{{<<tagger source:TagsDB>>}}} <<tagger source:TagDataBase>>
*You can also display ONLY the current tiddler tags
**{{{<<tagger taglist:false>>}}} <<tagger taglist:false>>
*To exclude tags from the list: {{{<<tagger exclude:"excludeLists Tag2 [[Tag with spaces]]">>}}} <<tagger exclude:"excludeLists Tag2 [[Tag with spaces]]">>
*For a custom button label: {{{<<tagger label:"custom label">>}}} <<tagger label:"custom label">>
*For a custom tooltip: {{{<<tagger tooltip:"custom tooltip">>}}} <<tagger tooltip:"custom tooltip">>
!CSS and Styling:
For those wishing to customize the popup appearance:
*the main popup has a class and id of popup has with all other popups.
*the nested tag popups have an id of nestedpopup
!Advanced Users:
You can change the global defaults for tagger, like the button label, the tags to exclude or whether to display the taglist or not, by editing the ''config.tagger.defaults'' section in the code.
!To Do:
*code optimization
*possibly a 'delete this tag' option.
!History
*version 1.0.1 (2006-06-01): fixed conflicts with QuickOpenTag (TagglyTagging) and AutoTagger.
/***
|''Name:''|~TaggerPlugin|
|''Version:''|1.0.1 (2006-06-01)|
|''Source:''|http://tw.lewcid.org//#TaggerPlugin|
|''Author:''|SaqImtiaz|
|''Description:''|Provides a drop down listing current tiddler tags, and allowing toggling of tags.|
|''Documentation:''|[[TaggerPluginDocumentation]]|
|''Source Code:''|[[TaggerPluginSource]]|
|''~TiddlyWiki:''|Version 2.0.8 or better|
***/
//{{{
config.tagger={
defaults:{
label: 'Tags: ',
tooltip: 'Manage tiddler tags',
taglist: 'true',
excludeTags: '',
notags: 'tiddler has no tags',
aretags: 'current tiddler tags:',
toggletext: 'add tags:'
}
};
config.macros.tagger={};
config.macros.tagger.arrow = (document.all?"▼":"▾"); // the fat one is the only one that works in IE
config.macros.tagger.handler = function(place,macroName,params,wikifier,paramString,tiddler) {
var defaults = config.tagger.defaults;
var nAV = paramString.parseParams('tagman', null, true);
var label = ((nAV[0].label)&&(nAV[0].label[0])!='.')?nAV[0].label[0]+this.arrow: defaults.label+this.arrow;
var tooltip = ((nAV[0].tooltip)&&(nAV[0].tooltip[0])!='.')?nAV[0].tooltip[0]: defaults.tooltip;
var taglist = ((nAV[0].taglist)&&(nAV[0].taglist[0])!='.')?nAV[0].taglist[0]: defaults.taglist;
var exclude = ((nAV[0].exclude)&&(nAV[0].exclude[0])!='.')?(nAV[0].exclude[0]).readBracketedList(): defaults.excludeTags.readBracketedList();
if ((nAV[0].source)&&(nAV[0].source[0])!='.')var source = nAV[0].source[0];
if (source&&!store.getTiddler(source)) return false;
var onclick = function(e) {
if (!e) var e = window.event;
var popup = Popup.create(this);
var tagsarray = store.getTags();
var tags=new Array();
for (var i=0; i<tagsarray.length; i++){
tags.push(tagsarray[i][0]);}
if (source)
{var sourcetiddler=store.getTiddler(source);
tags=sourcetiddler.tags.sort();}
var currentTags = tiddler.tags.sort();
var createButtons=function(text,theTag,tooltipPrefix){
var sp = createTiddlyElement(createTiddlyElement(popup,"li"),"span",null,"tagger");
var theToggle = createTiddlyButton(sp,text,tooltipPrefix+" '"+theTag+"'",taggerOnToggle,"button","toggleButton");
theToggle.setAttribute("tiddler",tiddler.title);
theToggle.setAttribute("tag",theTag);
insertSpacer(sp);
if (window.createTagButton_orig_mptw)
createTagButton_orig_mptw(sp,theTag);
else
createTagButton(sp,theTag);
}
createTiddlyElement(popup,"li",null,"listTitle",(tiddler.tags.length == 0 ? defaults.notags : defaults.aretags));
for (var t=0; t<currentTags.length; t++){
createButtons("[x]",currentTags[t],"remove tag ");
}
createTiddlyElement(createTiddlyElement(popup,"li"),"hr");
if (taglist!='false')
{ createTiddlyElement(popup,"li",null,"listTitle",defaults.toggletext);
for (var i=0; i<tags.length; i++){
if (!tiddler.tags.contains(tags[i])&&!exclude.contains(tags[i]))
{createButtons("[ ]",tags[i],"add tag ");
}
}
createTiddlyElement(createTiddlyElement(popup,"li"),"hr");
}
var newTagButton = createTiddlyButton(createTiddlyElement(popup,"li"),("Create new tag"),null,taggerOnToggle);
newTagButton.setAttribute("tiddler",tiddler.title);
if (source) newTagButton.setAttribute("source",source);
Popup.show(popup,false);
e.cancelBubble = true;
if (e.stopPropagation) e.stopPropagation();
return(false);
};
createTiddlyButton(place,label,tooltip,onclick,"button","taggerDrpBtn");
};
window.taggerOnToggle = function(e) {
var tag = this.getAttribute("tag");
var title = this.getAttribute("tiddler");
var tiddler = store.getTiddler(title);
if (!tag)
{
var newtag=prompt("Enter new tag:","");
if (newtag!=''&&newtag!=null)
{
var tag=newtag;
if (this.getAttribute("source"))
{var sourcetiddler = store.getTiddler(this.getAttribute("source"));
sourcetiddler.tags.pushUnique(newtag);}
}
else
{return false;};
}
if (!tiddler || !tiddler.tags)
{store.saveTiddler(title,title,'',config.options.txtUserName,new Date(),tag);}
else
{if (tiddler.tags.find(tag)==null)
{tiddler.tags.push(tag)}
else if(!newtag)
{tiddler.tags.splice(tiddler.tags.find(tag),1)};
store.saveTiddler(tiddler.title,tiddler.title,tiddler.text,tiddler.modifier,tiddler.modified,tiddler.tags);};
story.refreshTiddler(title,null,true);
if(config.options.chkAutoSave)
saveChanges();
return false;
};
setStylesheet(
".tagger a.button {font-weight: bold;display:inline; padding:0px;}\n"+
".tagger #toggleButton {padding-left:2px; padding-right:2px; margin-right:1px; font-size:110%;}\n"+
"#nestedtagger {background:#2E5ADF; border: 1px solid #0331BF;}\n"+
".popup .listTitle {color:#000;}\n"+
"",
"TaggerStyles");
window.lewcidTiddlerSwapTag = function (tiddler, oldTag, newTag){
for (var i = 0; i < tiddler.tags.length; i++)
if (tiddler.tags[i] == oldTag) {
tiddler.tags[i] = newTag;
return true;}
return false;
}
window.lewcidRenameTag = function(e) {
var tag=this.getAttribute("tag");
var newtag=prompt("Rename tag '"+tag+"' to:",tag);
if ((newtag==tag)||(newtag==null)) {return false;}
if(store.tiddlerExists(newtag))
{if(confirm(config.messages.overwriteWarning.format([newtag.toString()])))
story.closeTiddler(newtag,false,false);
else
return null;}
tagged=store.getTaggedTiddlers(tag);
if (tagged.length!=0){
for (var j = 0; j < tagged.length; j++)
lewcidTiddlerSwapTag(tagged[j],tag,newtag);}
if (store.tiddlerExists(tag))
{store.saveTiddler(tag,newtag);}
if (document.getElementById("tiddler"+tag))
{var oldTagTiddler = document.getElementById(story.idPrefix + tag);
var before= story.positionTiddler(oldTagTiddler);
var place = document.getElementById(story.container);
story.closeTiddler(tag,false,false);
story.createTiddler(place,before,newtag,null);
story.saveTiddler(newtag);}
if(config.options.chkAutoSave)
saveChanges();
return false;
}
window.onClickTag=function(e)
{
if (!e) var e = window.event;
var theTarget = resolveTarget(e);
var nested = (!isNested(theTarget));
if ((Popup.stack.length > 1)&&(nested==true)) {Popup.removeFrom(1);}
else if(Popup.stack.length > 0 && nested==false) {Popup.removeFrom(0);};
var theId = (nested==false)? "popup" : "nestedtagger";
var popup = createTiddlyElement(document.body,"ol",theId,"popup",null);
Popup.stack.push({root: this, popup: popup});
var tag = this.getAttribute("tag");
var title = this.getAttribute("tiddler");
if(popup && tag)
{
var tagged = store.getTaggedTiddlers(tag);
var titles = [];
var li,r;
for(r=0;r<tagged.length;r++)
if(tagged[r].title != title)
titles.push(tagged[r].title);
var lingo = config.views.wikified.tag;
if(titles.length > 0)
{
var openAll = createTiddlyButton(createTiddlyElement(popup,"li"),lingo.openAllText.format([tag]),lingo.openAllTooltip,onClickTagOpenAll);
openAll.setAttribute("tag",tag);
createTiddlyElement(createTiddlyElement(popup,"li"),"hr");
for(r=0; r<titles.length; r++)
{
createTiddlyLink(createTiddlyElement(popup,"li"),titles[r],true);
}
}
else
createTiddlyText(createTiddlyElement(popup,"li",null,"disabled"),lingo.popupNone.format([tag]));
createTiddlyElement(createTiddlyElement(popup,"li"),"hr");
var h = createTiddlyLink(createTiddlyElement(popup,"li"),tag,false);
createTiddlyText(h,lingo.openTag.format([tag]));
createTiddlyElement(createTiddlyElement(popup,"li"),"hr");
var renameTagButton = createTiddlyButton(createTiddlyElement(popup,"li"),("Rename tag '"+tag+"'"),null,lewcidRenameTag);
renameTagButton.setAttribute("tag",tag)
}
Popup.show(popup,false);
e.cancelBubble = true;
if (e.stopPropagation) e.stopPropagation();
return(false);
}
if (!window.isNested)
window.isNested = function(e) {
while (e != null) {
var contentWrapper = document.getElementById("contentWrapper");
if (contentWrapper == e) return true;
e = e.parentNode;
}
return false;
};
config.shadowTiddlers.TaggerPluginDocumentation="The documentation is available [[here.|http://tw.lewcid.org/#TaggerPluginDocumentation]]";
config.shadowTiddlers.TaggerPluginSource="The uncompressed source code is available [[here.|http://tw.lewcid.org/#TaggerPluginSource]]";
//}}}
/***
|''Name:''|TagsTreePlugin|
|''Description:''|Displays tags hierachy as a tree of tagged tiddlers.<br>Can be used to create dynamic outline navigation.|
|''Version:''|1.0.1|
|''Date:''|Jan 04,2008|
|''Source:''|http://visualtw.ouvaton.org/VisualTW.html|
|''Author:''|Pascal Collin|
|''License:''|[[BSD open source license|License]]|
|''~CoreVersion:''|2.1.0|
|''Browser:''|Firefox 2.0; InternetExplorer 6.0|
!Demo
On the plugin [[homepage|http://visualtw.ouvaton.org/VisualTW.html]] :
*Try to tag some <<newTiddler>> with a tag displayed in the menu and edit MainMenu.
*Look at some tags like [[Plugins]] or [[menu]].
!Installation
#import the plugin,
#save and reload,
#optionally, edit TagsTreeStyleSheet.
! Usage
{{{<<tagsTree>>}}} macro accepts the following //optional// parameters.
|!#|!parameter|!description|!by default|
|1|{{{root}}}|Uses {{{root}}} tag as tree root|- In a //tiddler// content or template : uses the tiddler as root tag.<br>- In the //page// content or template (by ex MainMenu) : displays all untagged tags.|
|2|{{{excludeTag}}}|Excludes all such tagged tiddlers from the tree|Uses default excludeLists tag|
|3|{{{level}}}|Expands nodes until level {{{level}}}.<br>Value {{{0}}} hides expand/collapse buttons.|Nodes are collapsed on first level|
|4|{{{depth}}}|Hierachy depth|6 levels depth (H1 to H6 header styles)|
|5|{{{sortField}}}|Alternate sort field. By example : "index".|Sorts tags and tiddlers alphabetically (on their title)|
|6|{{{labelField}}}|Alertnate label field. By example : "label".|Displays tiddler's title|
!Useful addons
*[[FieldsEditorPlugin]] : //create//, //edit//, //view// and //delete// commands in toolbar <<toolbar fields>>.
*[[TaggerPlugin]] : Provides a drop down listing current tiddler tags, and allowing toggling of tags.
!Advanced Users
You can change the global defaults for TagsTreePlugin, like default {{{level}}} value or level styles, by editing or overriding the first config.macros.tagsTree attributes below.
!Code
***/
//{{{
config.macros.tagsTree = {
expand : "+",
collapse : "–",
depth : 6,
level : 1,
sortField : "",
labelField : "",
styles : ["h1","h2","h3","h4","h5","h6"],
trees : {}
}
config.macros.tagsTree.handler = function(place,macroName,params,wikifier,paramString,tiddler)
{
var root = params[0] ? params[0] : (tiddler ? tiddler.title : null);
var excludeTag = params[1] ? params[1] : "excludeTagsTree";
var level = params[2] ? params[2] : config.macros.tagsTree.level;
var depth = params[3] ? params[3] : config.macros.tagsTree.depth;
var sortField = params[4] ? params[4] : config.macros.tagsTree.sortField;
var labelField = params[5] ? params[5] : config.macros.tagsTree.labelField;
var showButtons = (level>0);
var id = config.macros.tagsTree.getId(place);
if (config.macros.tagsTree.trees[id]==undefined) config.macros.tagsTree.trees[id]={};
config.macros.tagsTree.createSubTree(place,id,root,excludeTag,[],level>0 ? level : 1,depth, sortField, labelField,showButtons);
}
config.macros.tagsTree.createSubTree = function(place, id, root, excludeTag, ancestors, level, depth, sortField, labelField,showButtons){
var childNodes = root ? this.getChildNodes(root, ancestors) : this.getRootTags(excludeTag);
var isOpen = (level>0) || (!showButtons);
if (root && this.trees[id][root]!=undefined) isOpen = this.trees[id][root];
if (root && ancestors.length) {
var t = store.getTiddler(root);
if (childNodes.length && depth>0) {
var wrapper = createTiddlyElement(place , this.styles[Math.min(Math.max(ancestors.length,1),6)-1],null,"branch");
if (showButtons) {
b = createTiddlyButton(wrapper, isOpen ? config.macros.tagsTree.collapse : config.macros.tagsTree.expand, null, config.macros.tagsTree.onClick);
b.setAttribute("treeId",id);
b.setAttribute("tiddler",root);
}
createTiddlyText(createTiddlyLink(wrapper, root),t&&labelField ? t.fields[labelField] ? t.fields[labelField] : root : root);
}
else
createTiddlyText(createTiddlyLink(place, root,false,"leaf"),t&&labelField ? t.fields[labelField] ? t.fields[labelField] : root : root);
}
if (childNodes.length && depth) {
var d = createTiddlyElement(place,"div",null,"subtree");
d.style.display= isOpen ? "block" : "none";
if (sortField)
childNodes.sort(function(a, b){
var fa=a.fields[sortField];
var fb=b.fields[sortField];
return (fa==undefined && fb==undefined) ? a.title < b.title ? -1 : a.title > b.title ? 1 : 0 : (fa==undefined && fb!=undefined) ? 1 :(fa!=undefined && fb==undefined) ? -1 : fa < fb ? -1 : fa > fb ? 1 : 0;
})
for (var cpt=0; cpt<childNodes.length; cpt++)
this.createSubTree(d, id, childNodes[cpt].title, excludeTag, ancestors.concat(root), level-1, depth-1, sortField, labelField, showButtons);
}
}
config.macros.tagsTree.onClick = function(e){
var id = this.getAttribute("treeId");
var tiddler = this.getAttribute("tiddler");
var n = this.parentNode.nextSibling;
var isOpen = n.style.display != "none";
if(config.options.chkAnimate && anim && typeof Slider == "function")
anim.startAnimating(new Slider(n,!isOpen,null,"none"));
else
n.style.display = isOpen ? "none" : "block";
this.firstChild.nodeValue = isOpen ? config.macros.tagsTree.expand : config.macros.tagsTree.collapse;
config.macros.tagsTree.trees[id][tiddler]=!isOpen;
return false;
}
config.macros.tagsTree.getChildNodes = function(root ,ancestors){
var childs = store.getTaggedTiddlers(root);
var result = new Array();
for (var cpt=0; cpt<childs.length; cpt++)
if (childs[cpt].title!=root && ancestors.indexOf(childs[cpt].title)==-1) result.push(childs[cpt]);
return result;
}
config.macros.tagsTree.getRootTags = function(excludeTag){
var tags = store.getTags(excludeTag);
tags.sort(function(a,b) {return a[0].toLowerCase() < b[0].toLowerCase() ? -1 : (a[0].toLowerCase() == b[0].toLowerCase() ? 0 : +1);});
var result = new Array();
for (var cpt=0; cpt<tags.length; cpt++) {
var t = store.getTiddler(tags[cpt][0]);
if (!t || t.tags.length==0) result.push(t ? t : {title:tags[cpt][0],fields:{}});
}
return result;
}
config.macros.tagsTree.getId = function(element){
while (!element.id && element.parentNode) element=element.parentNode;
return element.id ? element.id : "<html>";
}
config.shadowTiddlers.TagsTreeStyleSheet = "/*{{{*/\n";
config.shadowTiddlers.TagsTreeStyleSheet +=".leaf, .subtree {display:block; margin-left : 0.5em}\n";
config.shadowTiddlers.TagsTreeStyleSheet +=".subtree {margin-bottom:0.5em}\n";
config.shadowTiddlers.TagsTreeStyleSheet +="#mainMenu {text-align:left}\n";
config.shadowTiddlers.TagsTreeStyleSheet +=".branch .button {border:1px solid #DDD; color:#AAA;font-size:9px;padding:0 2px;margin-right:0.3em;vertical-align:middle;text-align:center;}\n";
config.shadowTiddlers.TagsTreeStyleSheet +="/*}}}*/";
store.addNotification("TagsTreeStyleSheet", refreshStyles);
config.shadowTiddlers.MainMenu="<<tagsTree>>"
config.shadowTiddlers.PageTemplate = config.shadowTiddlers.PageTemplate.replace(/id='mainMenu' refresh='content' /,"id='mainMenu' refresh='content' force='true' ")
//}}}
[img[http://www.uni-saarland.de/fileadmin/templates/extranet_uds/gif/allgemein/logo_uds_175_49_neu.png]] http://www.coli.uni-saarland.de/~thorsten/tnt/
;Input :
:tokenized : one token per line
;Ouput :
:- basic : the tagger adds a second column to each line, containing the tag for the word.
: {{{FORM POS}}}
:- optionally: the tagger emits alternative tags for each token, together with a probability distribution
: {{{FORM POS POS1 PROB1 POS2 PROB2 ...}}}
;Trainning data :
:one token per line, the first column is the word, the second column is the tag.
: {{{FORM POS}}}
;Tag set :
:DE: The German model is trained on the Saarbrücker [[German newspaper corpus|http://www.coli.uni-sb.de/sfb378/negra-corpus/]] using the [[Stuttgart-Tübingen-Tagset|http://www.sfs.nphil.uni-tuebingen.de/Elwis/stts/stts.html]].
:EN: [[Susanne Corpus|http://www.grsampson.net/RSue.html]] / [[Penn Treebank|http://www.cis.upenn.edu/~treebank/]]
<<<
What is TnT?
TnT, the short form of Trigrams'n'Tags, is a very efficient statistical part-of-speech tagger that is trainable on different languages and virtually any tagset. The component for parameter generation trains on tagged corpora. The system incorporates several methods of smoothing and of handling unknown words.
TnT is not optimized for a particular language. Instead, it is optimized for training on a large variety of corpora. Adapting the tagger to a new language, new domain, or new tagset is very easy. Additionally, TnT is optimized for speed.
The tagger is an implementation of the Viterbi algorithm for second order Markov models. The main paradigm used for smoothing is linear interpolation, the respective weights are determined by deleted interpolation. Unknown words are handled by a suffix trie and successive abstraction.
<<<
/***
|''Name:''|ToggleSideBarTB|
|''Description''|allows to toggle left and right sidebar|
|''Version:''|1.1|
|''Type''|macro|
|''Author:''|[[TobiasBeer]]|
|''Info:''|http://tbgtd.tiddlyspot.com/#ToggleSideBarTB|
|''Source:''|http://tbgtd.tiddlyspot.com/#ToggleSideBarTB|
|''License''|[[Creative Commons Attribution-Share Alike 3.0|http://creativecommons.org/licenses/by-sa/3.0/]]|
|''~CoreVersion''|2.xx|
!Code
***/
//{{{
config.macros.toggleSideBarTB={
settings:{
arr1:"►",arr2:"◄",
lblToggle:"toggle %0",
leftBarID:"mainMenu",leftBarLbl:"main menu",
leftBarHide:"margin-left:1em;",leftBarShow:"margin-left:17.1em;",
rightBarID:"sidebar",rightBarLbl:"sidebar",
rightBarHide:"margin-right:1em;",rightBarShow:"margin-right:17.1em;"
},
handler:function(place,macroName,params,wikifier,paramString,tiddler){
var l=params[0]&¶ms[0]=="left";
var h=params[1]&¶ms[1]=="hide";
var no=params[2]&¶ms[2]=="notip";
var s=this.settings;
var el=l?s.leftBarID:s.rightBarID;
var btnId="ToggleSideBar"+(l?"Left":"Right");
var arr=l?(h?s.arr1:s.arr2):(h?s.arr2:s.arr1);
var lbl=(no?'':s.lblToggle.format([(l?s.leftBarLbl:s.rightBarLbl)]));
var fct=function(){config.macros.toggleSideBarTB.toggle(btnId)};
document.getElementById(el).setAttribute("toggle","hide");
createTiddlyButton(place,arr,lbl,fct,"button HideSideBarButton",btnId);
if(h)config.macros.toggleSideBarTB.toggle(btnId);
},
toggle:function(btnId){
var btn=document.getElementById(btnId);
var l=btn.id=="ToggleSideBarLeft";
var s=config.macros.toggleSideBarTB.settings;
var bl=document.getElementById(s.leftBarID);
var br=document.getElementById(s.rightBarID);
var bar=(l?bl:br);
var hl=bl.getAttribute("toggle")=='show';if(l)hl=!hl;
var hr=br.getAttribute("toggle")=='show';if(!l)hr=!hr;
var h=(l?hl:hr);
setStylesheet("#contentWrapper #displayArea {"+
(hl?s.leftBarHide:s.leftBarShow)+
(hr?s.rightBarHide:s.rightBarShow)+
"}","ToggleSideBarStyles");
bar.style.display=h?"none":"block";
bar.setAttribute("toggle",(h?"show":"hide"));
arr1=l?s.arr2:s.arr1;arr2=l?s.arr1:s.arr2;
btn.innerHTML=h?arr2:arr1;
}
}
//}}}
découpe en token du texte
|~ViewToolbar|closeTiddler closeOthers +editTiddler fields > syncing permalink references jump|
|~EditToolbar|+saveTiddler -cancelTiddler deleteTiddler|
http://www.ims.uni-stuttgart.de/projekte/corplex/TreeTagger/
;Input :
:raw text
;Output :
:{{{WORD POS LEMMA}}}
;Tag sets :
:EN: [[Penn Treebank English POS tag set]]
:FR: [[French TreeBank POS Tags]]
> The TreeTagger is a tool for annotating text with part-of-speech and lemma information. It was developed by Helmut Schmid in the TC project at the Institute for Computational Linguistics of the University of Stuttgart. The TreeTagger has been successfully used to tag German, English, French, Italian, Dutch, Spanish, Bulgarian, Russian, Greek, Portuguese, Chinese, Swahili, Latin, Estonian and old French texts and is adaptable to other languages if a lexicon and a manually tagged training corpus are available.
> The TreeTagger can also be used as a chunker for English, German, and French. The parameter file for the French chunker was kindly provided by Michel Généreux.
<<<
The tagger is described in the following two papers:
* Helmut Schmid (1995): [[Improvements in Part-of-Speech Tagging with an Application to German|ftp://ftp.ims.uni-stuttgart.de/pub/corpora/tree-tagger2.pdf]]. Proceedings of the ACL SIGDAT-Workshop. Dublin, Ireland.
* Helmut Schmid (1994): [[Probabilistic Part-of-Speech Tagging Using Decision Trees|ftp://ftp.ims.uni-stuttgart.de/pub/corpora/tree-tagger1.pdf]]. Proceedings of International Conference on New Methods in Language Processing, Manchester, UK.
<<<
''Links'' :
- //TreeTagger for Java// (tt4j) is a Java wrapper around the popular TreeTagger package by Helmut Schmid : http://code.google.com/p/tt4j/
/***
Description: Contains the stuff you need to use Tiddlyspot
Note, you also need UploadPlugin, PasswordOptionPlugin and LoadRemoteFileThroughProxy
from http://tiddlywiki.bidix.info for a complete working Tiddlyspot site.
***/
//{{{
// edit this if you are migrating sites or retrofitting an existing TW
config.tiddlyspotSiteId = 'french-postaggers';
// make it so you can by default see edit controls via http
config.options.chkHttpReadOnly = false;
window.readOnly = false; // make sure of it (for tw 2.2)
window.showBackstage = true; // show backstage too
// disable autosave in d3
if (window.location.protocol != "file:")
config.options.chkGTDLazyAutoSave = false;
// tweak shadow tiddlers to add upload button, password entry box etc
with (config.shadowTiddlers) {
SiteUrl = 'http://'+config.tiddlyspotSiteId+'.tiddlyspot.com';
SideBarOptions = SideBarOptions.replace(/(<<saveChanges>>)/,"$1<<tiddler TspotSidebar>>");
OptionsPanel = OptionsPanel.replace(/^/,"<<tiddler TspotOptions>>");
DefaultTiddlers = DefaultTiddlers.replace(/^/,"[[WelcomeToTiddlyspot]] ");
MainMenu = MainMenu.replace(/^/,"[[WelcomeToTiddlyspot]] ");
}
// create some shadow tiddler content
merge(config.shadowTiddlers,{
'TspotControls':[
"| tiddlyspot password:|<<option pasUploadPassword>>|",
"| site management:|<<upload http://" + config.tiddlyspotSiteId + ".tiddlyspot.com/store.cgi index.html . . " + config.tiddlyspotSiteId + ">>//(requires tiddlyspot password)//<br>[[control panel|http://" + config.tiddlyspotSiteId + ".tiddlyspot.com/controlpanel]], [[download (go offline)|http://" + config.tiddlyspotSiteId + ".tiddlyspot.com/download]]|",
"| links:|[[tiddlyspot.com|http://tiddlyspot.com/]], [[FAQs|http://faq.tiddlyspot.com/]], [[blog|http://tiddlyspot.blogspot.com/]], email [[support|mailto:support@tiddlyspot.com]] & [[feedback|mailto:feedback@tiddlyspot.com]], [[donate|http://tiddlyspot.com/?page=donate]]|"
].join("\n"),
'TspotOptions':[
"tiddlyspot password:",
"<<option pasUploadPassword>>",
""
].join("\n"),
'TspotSidebar':[
"<<upload http://" + config.tiddlyspotSiteId + ".tiddlyspot.com/store.cgi index.html . . " + config.tiddlyspotSiteId + ">><html><a href='http://" + config.tiddlyspotSiteId + ".tiddlyspot.com/download' class='button'>download</a></html>"
].join("\n"),
'WelcomeToTiddlyspot':[
"This document is a ~TiddlyWiki from tiddlyspot.com. A ~TiddlyWiki is an electronic notebook that is great for managing todo lists, personal information, and all sorts of things.",
"",
"@@font-weight:bold;font-size:1.3em;color:#444; //What now?// @@ Before you can save any changes, you need to enter your password in the form below. Then configure privacy and other site settings at your [[control panel|http://" + config.tiddlyspotSiteId + ".tiddlyspot.com/controlpanel]] (your control panel username is //" + config.tiddlyspotSiteId + "//).",
"<<tiddler TspotControls>>",
"See also GettingStarted.",
"",
"@@font-weight:bold;font-size:1.3em;color:#444; //Working online// @@ You can edit this ~TiddlyWiki right now, and save your changes using the \"save to web\" button in the column on the right.",
"",
"@@font-weight:bold;font-size:1.3em;color:#444; //Working offline// @@ A fully functioning copy of this ~TiddlyWiki can be saved onto your hard drive or USB stick. You can make changes and save them locally without being connected to the Internet. When you're ready to sync up again, just click \"upload\" and your ~TiddlyWiki will be saved back to tiddlyspot.com.",
"",
"@@font-weight:bold;font-size:1.3em;color:#444; //Help!// @@ Find out more about ~TiddlyWiki at [[TiddlyWiki.com|http://tiddlywiki.com]]. Also visit [[TiddlyWiki.org|http://tiddlywiki.org]] for documentation on learning and using ~TiddlyWiki. New users are especially welcome on the [[TiddlyWiki mailing list|http://groups.google.com/group/TiddlyWiki]], which is an excellent place to ask questions and get help. If you have a tiddlyspot related problem email [[tiddlyspot support|mailto:support@tiddlyspot.com]].",
"",
"@@font-weight:bold;font-size:1.3em;color:#444; //Enjoy :)// @@ We hope you like using your tiddlyspot.com site. Please email [[feedback@tiddlyspot.com|mailto:feedback@tiddlyspot.com]] with any comments or suggestions."
].join("\n")
});
//}}}
[img[http://uima.apache.org/images/UIMA_banner2tlpTm.png]] Unstructured Information Management Applications
Article du blog UIAM-FR pour construire les modèle français d'un POS tagger HMM : [[Construire des modélisations du French Treebank pour le UIMA HMM Tagger|http://www.uima-fr.org/planet//index.php?post_id=71]]
| !date | !user | !location | !storeUrl | !uploadDir | !toFilename | !backupdir | !origin |
| 01/10/2013 13:18:11 | french-postaggers | [[index.html|http://french-postaggers.tiddlyspot.com/index.html]] | [[store.cgi|http://french-postaggers.tiddlyspot.com/store.cgi]] | . | [[index.html | http://french-postaggers.tiddlyspot.com/index.html]] | . | ok |
| 01/10/2013 13:18:51 | french-postaggers | [[index.html|http://french-postaggers.tiddlyspot.com/index.html]] | [[store.cgi|http://french-postaggers.tiddlyspot.com/store.cgi]] | . | [[index.html | http://french-postaggers.tiddlyspot.com/index.html]] | . |
| 01/10/2013 13:19:42 | french-postaggers | [[index.html|http://french-postaggers.tiddlyspot.com/index.html]] | [[store.cgi|http://french-postaggers.tiddlyspot.com/store.cgi]] | . | [[index.html | http://french-postaggers.tiddlyspot.com/index.html]] | . |
| 01/10/2013 13:20:31 | french-postaggers | [[index.html|http://french-postaggers.tiddlyspot.com/index.html]] | [[store.cgi|http://french-postaggers.tiddlyspot.com/store.cgi]] | . | [[index.html | http://french-postaggers.tiddlyspot.com/index.html]] | . |
| 01/10/2013 13:23:06 | french-postaggers | [[french-postaggers.html|file:///home/montcheuil/Work/docs/POS-Taggers/french-postaggers.html]] | [[store.cgi|http://french-postaggers.tiddlyspot.com/store.cgi]] | . | [[index.html | http://french-postaggers.tiddlyspot.com/index.html]] | . |
| 01/10/2013 13:34:31 | french-postaggers | [[french-postaggers.html|file:///home/montcheuil/Work/docs/POS-Taggers/french-postaggers.html]] | [[store.cgi|http://french-postaggers.tiddlyspot.com/store.cgi]] | . | [[index.html | http://french-postaggers.tiddlyspot.com/index.html]] | . |
| 01/10/2013 14:00:41 | french-postaggers | [[french-postaggers.html|file:///home/montcheuil/Work/docs/POS-Taggers/french-postaggers.html]] | [[store.cgi|http://french-postaggers.tiddlyspot.com/store.cgi]] | . | [[index.html | http://french-postaggers.tiddlyspot.com/index.html]] | . |
| 01/10/2013 14:01:26 | french-postaggers | [[french-postaggers.html|file:///home/montcheuil/Work/docs/POS-Taggers/french-postaggers.html]] | [[store.cgi|http://french-postaggers.tiddlyspot.com/store.cgi]] | . | [[index.html | http://french-postaggers.tiddlyspot.com/index.html]] | . |
| 01/10/2013 16:29:15 | french-postaggers | [[french-postaggers.html|file:///home/montcheuil/Work/docs/POS-Taggers/french-postaggers.html]] | [[store.cgi|http://french-postaggers.tiddlyspot.com/store.cgi]] | . | [[index.html | http://french-postaggers.tiddlyspot.com/index.html]] | . | ok |
| 04/11/2013 15:10:00 | french-postaggers | [[french-postaggers.html|file:///home/montcheuil/Work/docs/POS-Taggers/french-postaggers.html]] | [[store.cgi|http://french-postaggers.tiddlyspot.com/store.cgi]] | . | [[index.html | http://french-postaggers.tiddlyspot.com/index.html]] | . |
/***
|''Name:''|UploadPlugin|
|''Description:''|Save to web a TiddlyWiki|
|''Version:''|4.1.3|
|''Date:''|Feb 24, 2008|
|''Source:''|http://tiddlywiki.bidix.info/#UploadPlugin|
|''Documentation:''|http://tiddlywiki.bidix.info/#UploadPluginDoc|
|''Author:''|BidiX (BidiX (at) bidix (dot) info)|
|''License:''|[[BSD open source license|http://tiddlywiki.bidix.info/#%5B%5BBSD%20open%20source%20license%5D%5D ]]|
|''~CoreVersion:''|2.2.0|
|''Requires:''|PasswordOptionPlugin|
***/
//{{{
version.extensions.UploadPlugin = {
major: 4, minor: 1, revision: 3,
date: new Date("Feb 24, 2008"),
source: 'http://tiddlywiki.bidix.info/#UploadPlugin',
author: 'BidiX (BidiX (at) bidix (dot) info',
coreVersion: '2.2.0'
};
//
// Environment
//
if (!window.bidix) window.bidix = {}; // bidix namespace
bidix.debugMode = false; // true to activate both in Plugin and UploadService
//
// Upload Macro
//
config.macros.upload = {
// default values
defaultBackupDir: '', //no backup
defaultStoreScript: "store.php",
defaultToFilename: "index.html",
defaultUploadDir: ".",
authenticateUser: true // UploadService Authenticate User
};
config.macros.upload.label = {
promptOption: "Save and Upload this TiddlyWiki with UploadOptions",
promptParamMacro: "Save and Upload this TiddlyWiki in %0",
saveLabel: "save to web",
saveToDisk: "save to disk",
uploadLabel: "upload"
};
config.macros.upload.messages = {
noStoreUrl: "No store URL in parmeters or options",
usernameOrPasswordMissing: "Username or password missing"
};
config.macros.upload.handler = function(place,macroName,params) {
if (readOnly)
return;
var label;
if (document.location.toString().substr(0,4) == "http")
label = this.label.saveLabel;
else
label = this.label.uploadLabel;
var prompt;
if (params[0]) {
prompt = this.label.promptParamMacro.toString().format([this.destFile(params[0],
(params[1] ? params[1]:bidix.basename(window.location.toString())), params[3])]);
} else {
prompt = this.label.promptOption;
}
createTiddlyButton(place, label, prompt, function() {config.macros.upload.action(params);}, null, null, this.accessKey);
};
config.macros.upload.action = function(params)
{
// for missing macro parameter set value from options
if (!params) params = {};
var storeUrl = params[0] ? params[0] : config.options.txtUploadStoreUrl;
var toFilename = params[1] ? params[1] : config.options.txtUploadFilename;
var backupDir = params[2] ? params[2] : config.options.txtUploadBackupDir;
var uploadDir = params[3] ? params[3] : config.options.txtUploadDir;
var username = params[4] ? params[4] : config.options.txtUploadUserName;
var password = config.options.pasUploadPassword; // for security reason no password as macro parameter
// for still missing parameter set default value
if ((!storeUrl) && (document.location.toString().substr(0,4) == "http"))
storeUrl = bidix.dirname(document.location.toString())+'/'+config.macros.upload.defaultStoreScript;
if (storeUrl.substr(0,4) != "http")
storeUrl = bidix.dirname(document.location.toString()) +'/'+ storeUrl;
if (!toFilename)
toFilename = bidix.basename(window.location.toString());
if (!toFilename)
toFilename = config.macros.upload.defaultToFilename;
if (!uploadDir)
uploadDir = config.macros.upload.defaultUploadDir;
if (!backupDir)
backupDir = config.macros.upload.defaultBackupDir;
// report error if still missing
if (!storeUrl) {
alert(config.macros.upload.messages.noStoreUrl);
clearMessage();
return false;
}
if (config.macros.upload.authenticateUser && (!username || !password)) {
alert(config.macros.upload.messages.usernameOrPasswordMissing);
clearMessage();
return false;
}
bidix.upload.uploadChanges(false,null,storeUrl, toFilename, uploadDir, backupDir, username, password);
return false;
};
config.macros.upload.destFile = function(storeUrl, toFilename, uploadDir)
{
if (!storeUrl)
return null;
var dest = bidix.dirname(storeUrl);
if (uploadDir && uploadDir != '.')
dest = dest + '/' + uploadDir;
dest = dest + '/' + toFilename;
return dest;
};
//
// uploadOptions Macro
//
config.macros.uploadOptions = {
handler: function(place,macroName,params) {
var wizard = new Wizard();
wizard.createWizard(place,this.wizardTitle);
wizard.addStep(this.step1Title,this.step1Html);
var markList = wizard.getElement("markList");
var listWrapper = document.createElement("div");
markList.parentNode.insertBefore(listWrapper,markList);
wizard.setValue("listWrapper",listWrapper);
this.refreshOptions(listWrapper,false);
var uploadCaption;
if (document.location.toString().substr(0,4) == "http")
uploadCaption = config.macros.upload.label.saveLabel;
else
uploadCaption = config.macros.upload.label.uploadLabel;
wizard.setButtons([
{caption: uploadCaption, tooltip: config.macros.upload.label.promptOption,
onClick: config.macros.upload.action},
{caption: this.cancelButton, tooltip: this.cancelButtonPrompt, onClick: this.onCancel}
]);
},
options: [
"txtUploadUserName",
"pasUploadPassword",
"txtUploadStoreUrl",
"txtUploadDir",
"txtUploadFilename",
"txtUploadBackupDir",
"chkUploadLog",
"txtUploadLogMaxLine"
],
refreshOptions: function(listWrapper) {
var opts = [];
for(i=0; i<this.options.length; i++) {
var opt = {};
opts.push();
opt.option = "";
n = this.options[i];
opt.name = n;
opt.lowlight = !config.optionsDesc[n];
opt.description = opt.lowlight ? this.unknownDescription : config.optionsDesc[n];
opts.push(opt);
}
var listview = ListView.create(listWrapper,opts,this.listViewTemplate);
for(n=0; n<opts.length; n++) {
var type = opts[n].name.substr(0,3);
var h = config.macros.option.types[type];
if (h && h.create) {
h.create(opts[n].colElements['option'],type,opts[n].name,opts[n].name,"no");
}
}
},
onCancel: function(e)
{
backstage.switchTab(null);
return false;
},
wizardTitle: "Upload with options",
step1Title: "These options are saved in cookies in your browser",
step1Html: "<input type='hidden' name='markList'></input><br>",
cancelButton: "Cancel",
cancelButtonPrompt: "Cancel prompt",
listViewTemplate: {
columns: [
{name: 'Description', field: 'description', title: "Description", type: 'WikiText'},
{name: 'Option', field: 'option', title: "Option", type: 'String'},
{name: 'Name', field: 'name', title: "Name", type: 'String'}
],
rowClasses: [
{className: 'lowlight', field: 'lowlight'}
]}
};
//
// upload functions
//
if (!bidix.upload) bidix.upload = {};
if (!bidix.upload.messages) bidix.upload.messages = {
//from saving
invalidFileError: "The original file '%0' does not appear to be a valid TiddlyWiki",
backupSaved: "Backup saved",
backupFailed: "Failed to upload backup file",
rssSaved: "RSS feed uploaded",
rssFailed: "Failed to upload RSS feed file",
emptySaved: "Empty template uploaded",
emptyFailed: "Failed to upload empty template file",
mainSaved: "Main TiddlyWiki file uploaded",
mainFailed: "Failed to upload main TiddlyWiki file. Your changes have not been saved",
//specific upload
loadOriginalHttpPostError: "Can't get original file",
aboutToSaveOnHttpPost: 'About to upload on %0 ...',
storePhpNotFound: "The store script '%0' was not found."
};
bidix.upload.uploadChanges = function(onlyIfDirty,tiddlers,storeUrl,toFilename,uploadDir,backupDir,username,password)
{
var callback = function(status,uploadParams,original,url,xhr) {
if (!status) {
displayMessage(bidix.upload.messages.loadOriginalHttpPostError);
return;
}
if (bidix.debugMode)
alert(original.substr(0,500)+"\n...");
// Locate the storeArea div's
var posDiv = locateStoreArea(original);
if((posDiv[0] == -1) || (posDiv[1] == -1)) {
alert(config.messages.invalidFileError.format([localPath]));
return;
}
bidix.upload.uploadRss(uploadParams,original,posDiv);
};
if(onlyIfDirty && !store.isDirty())
return;
clearMessage();
// save on localdisk ?
if (document.location.toString().substr(0,4) == "file") {
var path = document.location.toString();
var localPath = getLocalPath(path);
saveChanges();
}
// get original
var uploadParams = new Array(storeUrl,toFilename,uploadDir,backupDir,username,password);
var originalPath = document.location.toString();
// If url is a directory : add index.html
if (originalPath.charAt(originalPath.length-1) == "/")
originalPath = originalPath + "index.html";
var dest = config.macros.upload.destFile(storeUrl,toFilename,uploadDir);
var log = new bidix.UploadLog();
log.startUpload(storeUrl, dest, uploadDir, backupDir);
displayMessage(bidix.upload.messages.aboutToSaveOnHttpPost.format([dest]));
if (bidix.debugMode)
alert("about to execute Http - GET on "+originalPath);
var r = doHttp("GET",originalPath,null,null,username,password,callback,uploadParams,null);
if (typeof r == "string")
displayMessage(r);
return r;
};
bidix.upload.uploadRss = function(uploadParams,original,posDiv)
{
var callback = function(status,params,responseText,url,xhr) {
if(status) {
var destfile = responseText.substring(responseText.indexOf("destfile:")+9,responseText.indexOf("\n", responseText.indexOf("destfile:")));
displayMessage(bidix.upload.messages.rssSaved,bidix.dirname(url)+'/'+destfile);
bidix.upload.uploadMain(params[0],params[1],params[2]);
} else {
displayMessage(bidix.upload.messages.rssFailed);
}
};
// do uploadRss
if(config.options.chkGenerateAnRssFeed) {
var rssPath = uploadParams[1].substr(0,uploadParams[1].lastIndexOf(".")) + ".xml";
var rssUploadParams = new Array(uploadParams[0],rssPath,uploadParams[2],'',uploadParams[4],uploadParams[5]);
var rssString = generateRss();
// no UnicodeToUTF8 conversion needed when location is "file" !!!
if (document.location.toString().substr(0,4) != "file")
rssString = convertUnicodeToUTF8(rssString);
bidix.upload.httpUpload(rssUploadParams,rssString,callback,Array(uploadParams,original,posDiv));
} else {
bidix.upload.uploadMain(uploadParams,original,posDiv);
}
};
bidix.upload.uploadMain = function(uploadParams,original,posDiv)
{
var callback = function(status,params,responseText,url,xhr) {
var log = new bidix.UploadLog();
if(status) {
// if backupDir specified
if ((params[3]) && (responseText.indexOf("backupfile:") > -1)) {
var backupfile = responseText.substring(responseText.indexOf("backupfile:")+11,responseText.indexOf("\n", responseText.indexOf("backupfile:")));
displayMessage(bidix.upload.messages.backupSaved,bidix.dirname(url)+'/'+backupfile);
}
var destfile = responseText.substring(responseText.indexOf("destfile:")+9,responseText.indexOf("\n", responseText.indexOf("destfile:")));
displayMessage(bidix.upload.messages.mainSaved,bidix.dirname(url)+'/'+destfile);
store.setDirty(false);
log.endUpload("ok");
} else {
alert(bidix.upload.messages.mainFailed);
displayMessage(bidix.upload.messages.mainFailed);
log.endUpload("failed");
}
};
// do uploadMain
var revised = bidix.upload.updateOriginal(original,posDiv);
bidix.upload.httpUpload(uploadParams,revised,callback,uploadParams);
};
bidix.upload.httpUpload = function(uploadParams,data,callback,params)
{
var localCallback = function(status,params,responseText,url,xhr) {
url = (url.indexOf("nocache=") < 0 ? url : url.substring(0,url.indexOf("nocache=")-1));
if (xhr.status == 404)
alert(bidix.upload.messages.storePhpNotFound.format([url]));
if ((bidix.debugMode) || (responseText.indexOf("Debug mode") >= 0 )) {
alert(responseText);
if (responseText.indexOf("Debug mode") >= 0 )
responseText = responseText.substring(responseText.indexOf("\n\n")+2);
} else if (responseText.charAt(0) != '0')
alert(responseText);
if (responseText.charAt(0) != '0')
status = null;
callback(status,params,responseText,url,xhr);
};
// do httpUpload
var boundary = "---------------------------"+"AaB03x";
var uploadFormName = "UploadPlugin";
// compose headers data
var sheader = "";
sheader += "--" + boundary + "\r\nContent-disposition: form-data; name=\"";
sheader += uploadFormName +"\"\r\n\r\n";
sheader += "backupDir="+uploadParams[3] +
";user=" + uploadParams[4] +
";password=" + uploadParams[5] +
";uploaddir=" + uploadParams[2];
if (bidix.debugMode)
sheader += ";debug=1";
sheader += ";;\r\n";
sheader += "\r\n" + "--" + boundary + "\r\n";
sheader += "Content-disposition: form-data; name=\"userfile\"; filename=\""+uploadParams[1]+"\"\r\n";
sheader += "Content-Type: text/html;charset=UTF-8" + "\r\n";
sheader += "Content-Length: " + data.length + "\r\n\r\n";
// compose trailer data
var strailer = new String();
strailer = "\r\n--" + boundary + "--\r\n";
data = sheader + data + strailer;
if (bidix.debugMode) alert("about to execute Http - POST on "+uploadParams[0]+"\n with \n"+data.substr(0,500)+ " ... ");
var r = doHttp("POST",uploadParams[0],data,"multipart/form-data; ;charset=UTF-8; boundary="+boundary,uploadParams[4],uploadParams[5],localCallback,params,null);
if (typeof r == "string")
displayMessage(r);
return r;
};
// same as Saving's updateOriginal but without convertUnicodeToUTF8 calls
bidix.upload.updateOriginal = function(original, posDiv)
{
if (!posDiv)
posDiv = locateStoreArea(original);
if((posDiv[0] == -1) || (posDiv[1] == -1)) {
alert(config.messages.invalidFileError.format([localPath]));
return;
}
var revised = original.substr(0,posDiv[0] + startSaveArea.length) + "\n" +
store.allTiddlersAsHtml() + "\n" +
original.substr(posDiv[1]);
var newSiteTitle = getPageTitle().htmlEncode();
revised = revised.replaceChunk("<title"+">","</title"+">"," " + newSiteTitle + " ");
revised = updateMarkupBlock(revised,"PRE-HEAD","MarkupPreHead");
revised = updateMarkupBlock(revised,"POST-HEAD","MarkupPostHead");
revised = updateMarkupBlock(revised,"PRE-BODY","MarkupPreBody");
revised = updateMarkupBlock(revised,"POST-SCRIPT","MarkupPostBody");
return revised;
};
//
// UploadLog
//
// config.options.chkUploadLog :
// false : no logging
// true : logging
// config.options.txtUploadLogMaxLine :
// -1 : no limit
// 0 : no Log lines but UploadLog is still in place
// n : the last n lines are only kept
// NaN : no limit (-1)
bidix.UploadLog = function() {
if (!config.options.chkUploadLog)
return; // this.tiddler = null
this.tiddler = store.getTiddler("UploadLog");
if (!this.tiddler) {
this.tiddler = new Tiddler();
this.tiddler.title = "UploadLog";
this.tiddler.text = "| !date | !user | !location | !storeUrl | !uploadDir | !toFilename | !backupdir | !origin |";
this.tiddler.created = new Date();
this.tiddler.modifier = config.options.txtUserName;
this.tiddler.modified = new Date();
store.addTiddler(this.tiddler);
}
return this;
};
bidix.UploadLog.prototype.addText = function(text) {
if (!this.tiddler)
return;
// retrieve maxLine when we need it
var maxLine = parseInt(config.options.txtUploadLogMaxLine,10);
if (isNaN(maxLine))
maxLine = -1;
// add text
if (maxLine != 0)
this.tiddler.text = this.tiddler.text + text;
// Trunck to maxLine
if (maxLine >= 0) {
var textArray = this.tiddler.text.split('\n');
if (textArray.length > maxLine + 1)
textArray.splice(1,textArray.length-1-maxLine);
this.tiddler.text = textArray.join('\n');
}
// update tiddler fields
this.tiddler.modifier = config.options.txtUserName;
this.tiddler.modified = new Date();
store.addTiddler(this.tiddler);
// refresh and notifiy for immediate update
story.refreshTiddler(this.tiddler.title);
store.notify(this.tiddler.title, true);
};
bidix.UploadLog.prototype.startUpload = function(storeUrl, toFilename, uploadDir, backupDir) {
if (!this.tiddler)
return;
var now = new Date();
var text = "\n| ";
var filename = bidix.basename(document.location.toString());
if (!filename) filename = '/';
text += now.formatString("0DD/0MM/YYYY 0hh:0mm:0ss") +" | ";
text += config.options.txtUserName + " | ";
text += "[["+filename+"|"+location + "]] |";
text += " [[" + bidix.basename(storeUrl) + "|" + storeUrl + "]] | ";
text += uploadDir + " | ";
text += "[[" + bidix.basename(toFilename) + " | " +toFilename + "]] | ";
text += backupDir + " |";
this.addText(text);
};
bidix.UploadLog.prototype.endUpload = function(status) {
if (!this.tiddler)
return;
this.addText(" "+status+" |");
};
//
// Utilities
//
bidix.checkPlugin = function(plugin, major, minor, revision) {
var ext = version.extensions[plugin];
if (!
(ext &&
((ext.major > major) ||
((ext.major == major) && (ext.minor > minor)) ||
((ext.major == major) && (ext.minor == minor) && (ext.revision >= revision))))) {
// write error in PluginManager
if (pluginInfo)
pluginInfo.log.push("Requires " + plugin + " " + major + "." + minor + "." + revision);
eval(plugin); // generate an error : "Error: ReferenceError: xxxx is not defined"
}
};
bidix.dirname = function(filePath) {
if (!filePath)
return;
var lastpos;
if ((lastpos = filePath.lastIndexOf("/")) != -1) {
return filePath.substring(0, lastpos);
} else {
return filePath.substring(0, filePath.lastIndexOf("\\"));
}
};
bidix.basename = function(filePath) {
if (!filePath)
return;
var lastpos;
if ((lastpos = filePath.lastIndexOf("#")) != -1)
filePath = filePath.substring(0, lastpos);
if ((lastpos = filePath.lastIndexOf("/")) != -1) {
return filePath.substring(lastpos + 1);
} else
return filePath.substring(filePath.lastIndexOf("\\")+1);
};
bidix.initOption = function(name,value) {
if (!config.options[name])
config.options[name] = value;
};
//
// Initializations
//
// require PasswordOptionPlugin 1.0.1 or better
bidix.checkPlugin("PasswordOptionPlugin", 1, 0, 1);
// styleSheet
setStylesheet('.txtUploadStoreUrl, .txtUploadBackupDir, .txtUploadDir {width: 22em;}',"uploadPluginStyles");
//optionsDesc
merge(config.optionsDesc,{
txtUploadStoreUrl: "Url of the UploadService script (default: store.php)",
txtUploadFilename: "Filename of the uploaded file (default: in index.html)",
txtUploadDir: "Relative Directory where to store the file (default: . (downloadService directory))",
txtUploadBackupDir: "Relative Directory where to backup the file. If empty no backup. (default: ''(empty))",
txtUploadUserName: "Upload Username",
pasUploadPassword: "Upload Password",
chkUploadLog: "do Logging in UploadLog (default: true)",
txtUploadLogMaxLine: "Maximum of lines in UploadLog (default: 10)"
});
// Options Initializations
bidix.initOption('txtUploadStoreUrl','');
bidix.initOption('txtUploadFilename','');
bidix.initOption('txtUploadDir','');
bidix.initOption('txtUploadBackupDir','');
bidix.initOption('txtUploadUserName','');
bidix.initOption('pasUploadPassword','');
bidix.initOption('chkUploadLog',true);
bidix.initOption('txtUploadLogMaxLine','10');
// Backstage
merge(config.tasks,{
uploadOptions: {text: "upload", tooltip: "Change UploadOptions and Upload", content: '<<uploadOptions>>'}
});
config.backstageTasks.push("uploadOptions");
//}}}
<!--{{{-->
<div class='toolbar' role='navigation'><span macro='toolbar [[ToolbarCommands::ViewToolbar]]'/><span macro='tagger'/></div>
<div class='title' macro='view title'></div>
<div class='subtitle'><span macro='view modifier link'></span>, <span macro='view modified date'></span> (<span macro='message views.wikified.createdPrompt'></span> <span macro='view created date'></span>)</div>
<div class='tagging' macro='tagging'></div>
<div class='tagged' macro='tags'></div>
<div class='viewer' macro='view text wikified'></div>
<div class='tagClear'></div>
<!--}}}-->
Tagset used by : <<list filter [tag[tags:WSJ]]>>
http://freecode.com/articles/processing-corpora-with-python-and-the-natural-language-toolkit
|CC |Coordinating conjunction |
|CD |Cardinal number |
|DT |Determiner |
|EX |Existential there |
|FW |Foreign word |
|IN |Preposition/subord. conjunction |
|JJ |Adjective |
|JJR |Adjective, comparative |
|JJS |Adjective, superlative |
|LS |List item marker |
|MD |Modal |
|NN |Noun, singular or mass |
|NNS |Noun, plural |
|NNP |Proper noun, singular |
|NNPS |Proper noun plural |
|PDT |Predeterminer |
|POS |Possessive ending |
|PRP |Personal pronoun |
|PP$ |Possessive pronoun |
|RB |Adverb |
|RBR |Adverb, comparative |
|RBS |Adverb, superlative |
|RP |Particle |
|SYM |Symbol (mathematical or scientific) |
|TO |to |
|UH |Interjection |
|VB |Verb, base form |
|VBD |Verb, past tense |
|VBG |Verb, gerund/present participle |
|VBN |Verb, past participle |
|VBP |Verb, non-3rd ps. sing. present |
|VBZ |Verb,3rd ps. sing. present |
|WDT |wh-determiner |
|WP |wh-pronoun |
|WP$ |Possessive wh-pronoun |
|WRB |wh-adverb |
Conditional random field [[(wikipedia)|http://en.wikipedia.org/wiki/Conditional_random_field]]
Decision Tree [[(wikipedia)|http://en.wikipedia.org/wiki/Decision_tree_learning]]
Hidden Markov model [[(wikipedia)|http://en.wikipedia.org/wiki/Hidden_Markov_model]]
Log-linear model [[(wikipedia)|http://en.wikipedia.org/wiki/Log-linear_model]]
Maximum-entropy Markov model [[(wikipedia)|http://en.wikipedia.org/wiki/Maximum-entropy_Markov_model]]
Maximum entropy classifier [[(wikipedia)|http://en.wikipedia.org/wiki/Maximum_entropy_classifier]]
Perceptron [[(wikipedia)|http://en.wikipedia.org/wiki/Perceptron]]
Support vector machine [[(wikipedia)|http://en.wikipedia.org/wiki/Support_vector_machine]]
{{{Pierre Vinken , 61 years old , will join the board as a nonexecutive director Nov. 29 .}}}
{{{Mr. Vinken is chairman of Elsevier N.V. , the Dutch publishing group .}}}
1 //token// par ligne, multicolones
1 //sentence// par ligne, //token//s (y compris les ponctuations) separés par des //espace//s <<slider show-in-sent-ex [[exemple in:sent]] "(exemple)" "exemple d'entrée tokenisée 1 phrase par ligne.">>
1 //token// par ligne, avec ou sans marque de fin de phrase (le plus souvent une ligne blanche)
{{{Expected text language: English}}}
{{{Working on example.txt...}}}
[img[http://opennlp.apache.org/images/onlplogo.jpg]] http://opennlp.apache.org/index.html
;Input :
:tokenized text : 1 sentence per line, token separated by space
;Output/Trainning data :
:append POS tags
: {{{Pierre_NNP Vinken_NNP ,_, 61_CD years_NNS old_JJ ,_, will_MD join_VB the_DT board_NN as_IN
a_DT nonexecutive_JJ director_NN Nov._NNP 29_CD ._.
Mr._NNP Vinken_NNP is_VBZ chairman_NN of_IN Elsevier_NNP N.V._NNP ,_, the_DT Dutch_NNP publishing_VBG group_NN}}}
>The Apache OpenNLP library is a machine learning based toolkit for the processing of natural language text.
>
>It supports the most common NLP tasks, such as tokenization, sentence segmentation, part-of-speech tagging, named entity extraction, chunking, parsing, and coreference resolution. These tasks are usually required to build more advanced text processing services. OpenNLP also includes maximum entropy and perceptron based machine learning.
Voir http://www.sygtext.fr/ (le site de référence du Système SYGMART©)
Jeu d'étiquettes utilisées par le tagger.
Le jeu d'étiquettes dépends généralement de la langue :
* pour le français : <<matchTags popup "label:tag-set & lang:FR" tag-set AND lang:FR>> <<matchTags "#[[%0]]" "\n" tag-set AND lang:FR>>
* pour l'anglais : <<matchTags popup "label:tag-set & lang:EN" tag-set AND lang:EN>> <<matchTags "#[[%0]]" "\n" tag-set AND lang:EN>>