HeLI-OTS 1.4

HeLI off-the-shelf language identifier with language models for 200 languages.

Usage:
java -jar HeLI.jar -r <infile> -w <outfile>

The program will read the <infile> and classify the language of each line as one of the 200 languages it knows
and writes the results, one ISO 639-3 code per line, into file <outfile>.

You can use the -c option to make the program print a confidence score for the identification after each language code.

Usage:
java -jar HeLI.jar -c -r <infile> -w <outfile>

You can give the list of comma-separated ISO 639-3 identifiers for relevant languages after -l option.

Usage:
java -jar HeLI.jar -r <infile> -w <outfile> -l fin,swe,eng

You can give the number of top-scored languages to print after the -t option. (overrides confidence)

Usage:
java -jar HeLI.jar -r <infile> -w <outfile> -l fin,swe,eng -t 2

If you omit both of the filenames, the program will read the standard input one line at a time and write the result to standard output.

It can identify c. 3000 sentences per second using one core on a 2021 laptop and around 3 gigabytes of memory.

If you use this program in producing scientific publications, please refer to: 
 @inproceedings{heliots2022,
     title = "{H}e{LI-OTS}, Off-the-shelf Language Identifier for Text",
     author = "Jauhiainen, Tommi  and
       Jauhiainen, Heidi  and
       Lind{\'e}n, Krister",
     booktitle = "Proceedings of the 13th Conference on Language Resources and Evaluation",
     month = june,
     year = "2022",
     address = "Marseille, France",
     publisher = "European Language Resources Association",
     url = "http://www.lrec-conf.org/proceedings/lrec2022/pdf/2022.lrec-1.416.pdf",
     pages = "3912--3922",
     language = "English",
 }

Producing and publishing this software has been partly supported by The Finnish Research Impact Foundation Tandem Industry Academia -funding in cooperation with Lingsoft.