Systems and methods for determining translation accuracy in multi-user multi-lingual communications

Patent Citations (211)

Non-Patent Literature (68)

• U.S. Appl. No. 13/763,565, filed Feb. 8, 2013, Systems and Methods for Multi-User Multi-Lingual Communications, Leydon, et al.Applicant
• U.S. Appl. No. 61/778,282, filed Mar. 12, 2013, Group Chat Translation Systems and Methods, Leydon, et al.Applicant
• U.S. Appl. No. 14/221,142, filed Mar. 20, 2014, Systems and Methods for Incentivizing User Feedback for Translation Processing, Leydon, et al.Applicant
• U.S. Appl. No. 14/517,183, filed Oct. 17, 2014, Systems and Methods for Language Detection, Bojja, et al.Applicant
• U.S. Appl. No. 14/985,946, filed Dec. 31, 2015, Systems and Methods for Multi-User Multi-Lingual Communications, Orsini, et al.Applicant
• U.S. Appl. No. 15/003,345, filed Jan. 21, 2016, Systems and Methods for Multi-User Multi-Lingual Communications, Orsini, et al.Applicant
• U.S. Appl. No. 15/076,249, filed Mar. 21, 2016, Systems and Methods for Incentivizing User Feedback for Translation Processing, Leydon.Applicant
• U.S. Appl. No. 15/003,345, filed Jan. 21, 2016, Systems and Methods for Determining Translation Accuracy in Multi-User Multi-Lingual Communications, Orsini, et al.Applicant
• Ahmed, et al., “Language Identification from Text Using n-gram Based Cumulative Frequency Addition,” in Proceedings of Student/Faculty Research Day, CSIS, Pace University; pp. 12.1-12.8; May 2004.Applicant
• Aikawa et al., “The Impact of Crowdsourcing Post-editing with the Collaborative Translation Framework,” JapTAL Oct. 22-24, 2012; LNAI; 7614:1-10.Applicant
• Ambati et al., “Collaborative Workflow for Crowdsourcing Translation,” Proc. of the ACM 2012 conf. on Computer Supported Cooperative Work, ACM; 1191-1194; Feb. 11-15, 2012.Applicant
• Baldwin, T. and Lui, M., “Language identification: The Long and the Short of the Matter,” in Proceedings of NAACL-HLT; pp. 229-237; Jun. 2010.Applicant
• Bergsma, et al., “Language Identification for Creating Language-specific Twitter Collections,” in Proceedings of the Second Workshop on Language in Social Media; pp. 65-74; Jun. 2012.Applicant
• Callison-Burch et al., “Creating Speech and Language Data with Amazon's Mechanical Turk”, Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk; 1-12, Jun. 6, 2010.Applicant
• Callison-Burch, C., “Fast, Cheap, and Creative: Evaluating Translation Quality Using Amazon's Mechanical Turk,” Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pp. 286-295, Singapore, Aug. 6-7, 2009.Applicant
• Carter, et al., “Microblog Language Identification: Overcoming the Limitations of Short, Unedited and Idiomatic Text,” Language Resources and Evaluation; 47(1): 195-215; Mar. 2013.Applicant
• Cavnar, W. and Trenkle, J., “N-gram-based Text Categorization,” in Proceedings of the Third Symposium on Document Analysis and Information Retrieval; Apr. 1994, 14 pgs.Applicant
• Ceylan, H. and Kim, Y., “Language Identification of Search Engine Queries,” in Proceedings of ACL-IJCNLP; 2:1066-1074; Aug. 2009.Applicant
• Chang, C. and Lin, C., “LIBSVM: A Library for Support Vector Machines,” ACM Transactions on Intelligent Systems and Technology, 2(27):1-39; Apr. 2011.Applicant
• Dunning, “Statistical Identification of Language,” Computing Research Laboratory, New Mexico State University; Mar. 1994, 31 pgs.Applicant
• Fan, et al., “LIBLINEAR: A Library for Large Linear Classification,” Journal of Machine Learning Research; 9:1871-1874; Aug. 2008.Applicant
• Foster, et al., “#hardtoparse: POS Tagging and Pursing the Twitterverse,” in Proceedings of the AAAI Workshop on Analyzing Microtext; Aug. 2011, 7 pgs.Applicant
• Gottron, T. and Lipka, N., “A Comparison of Language Identification Approaches on Short, Query-style Texts,” in Advances in Information Retrieval; pp. 611-614; Mar. 2010.Applicant
• Grothe, et al., “A Comparative Study on Language Identification Methods,” in Proceedings of LREC; pp. 980-985; May 2008.Applicant
• Hughes, et al., “Reconsidering Language Identification for Written Language Resources,” in Proceedings of LREC; pp. 485-488; May 2006.Applicant
• Hulin et al., “Applications of Item Response Theory to Analysis of Attitude Scale Translations,” American Psychological Association; vol. 67(6); Dec. 1982; 51 pgs.Applicant
• Int'l Search Report and Written Opinion of the ISA/US in PCT/US2014/015632; Jul. 8, 2014; 8 pgs.Applicant
• Partial Int'l Search Report of the ISA/EP in PCT/US2014/040676; Feb. 17, 2015; 5 pgs.Applicant
• Int'l Search Report and Written Opinion of the ISA/US in PCT/US2014/040676; May 6, 2015; 16 pgs.Applicant
• Int'l Search Report and Written Opinion of the ISA/US in PCT/US2014/061141; Jun. 16, 2015; 13pgs.Applicant
• Little, G., “Turkit: Tools for Iterative Tasks on Mechanical Turk,” IEEE Symposium on Visual Languages and Human-Centric Computing; pp. 252-253; Sep. 20, 2009.Applicant
• Liu, et al., “A Broad-coverage Normalization System for Social Media Language,” in Proceedings of ACL; pp. 1035-1044; Jul. 2012.Applicant
• Liu, et al., “Recognizing Named Entities in Tweets,” in Proceedings of ACL-HLT; 1:359-367; Jun. 2011.Applicant
• Lui, M. and Baldwin, T., “Accurate Language Identification of Twitter Messages,” Proceedings of the 5th Workshop on Language Analysis for Social Media (LASM)@EACL 2014; pp. 17-25; Apr. 26-30, 2014.Applicant
• Lui, et al., “Automatic Detection and Language Identification of Multilingual Documents,” Transactions of the Association for Computational Linguistics, 2:27-40; Feb. 2014.Applicant
• Lui, M. and Baldwin, T., “Cross-domain Feature Selection for Language Identification,” Proceedings of the 5th International Joint Conference on Natural Language Processing; pp. 553-561; Nov. 8-13, 2011.Applicant
• Lui, M. and Baldwin, T., “Iangid.py: An Off-the-shelf Language Identification Tool,” Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics; pp. 25-30; Jul. 8-14, 2012.Applicant
• Monteith, et al., “Turning Bayesian Model Averaging Into Bayesian Model Combination,” Proceedings of the International Joint Conference on Neural Networks IJCNN'11; Jul. 31-Aug. 5, 2011; 7pgs.Applicant
• Papineni, K., et al. “BLEU: A Method for Automatic Evaluation of Machine Translation,” Proc. 40th annual Meeting on Assoc. for Computational Linguistics (ACL); Jul. 2002; pp. 311-318.Applicant
• Popovic, et al., “Syntax-oriented Evaluation Measures for Machine Translation Output,” Proc. of the Fourth Workshop on Statistical Machine Translation, pp. 29-32, Mar. 30-31, 2009.Applicant
• Qureshi et al., Collusion Detection and Prevention with Fire+ Trust and Reputation Model, 2010, IEEE, Computer and Information Technology (CIT), 2010 IEEE 10th International Conference, pp. 2548-2555; Jun. 2010.Applicant
• Ritter, et al., “Named Entity Recognition in Tweets: An Experimental Study,” in Proceedings of EMNLP;pp. 1524-1534; Jul. 2011.Applicant
• Rouse, M., “Parallel Processing,” Search Data Center.com; Mar. 27, 2007; 2pgs.Applicant
• Shieber, S.M., and Nelken R., “Abbreviated Text Input Using Language Modeling.” Natural Language Eng; 13(2):165-183; Jun. 2007.Applicant
• Tromp, E. and Pechenizkiy, M., “Graph-based n-gram Language Identification on Short Texts,” in Proceedings of the 20th Machine Learning Conference of Belgium and the Netherlands; May 2011; 8 pgs.Applicant
• Vatanen, et al., “Language Identification of Short Text Segments with n-gram Models,” in Proceedings of LREC; pp. 3423-3430; May 2010.Applicant
• Vogel, J. and Tresner-Kirsch, D., “Robust Language Identification in Short, Noisy Texts: Improvements to LIGA,” in Proceedings of the 3rd International Workshop on Mining Ubiquitous and Social Environments; pp. 1-9; Jul. 2012.Applicant
• Xia, F. and Lewis, W.D., “Applying NLP Technologies to the Collection and Enrichment of Language Data on the Web to Aid Linguistic Research,” Proc. of the EACL 2009 Workshop on Language Tech. and Resources for Cultural Heritage, Social Sciences, Humanities, and Education—LaTech—SHELT&R 2009; pp. 51-59; Mar. 2009.Applicant
• Zaidan et al., “Crowdsourcing Translation: Professional Quality from Non-Professionals,” Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pp. 1220-1229, Portland, Oregon, Jun. 19-24, 2011.Applicant
• “Arabic script in Unicode,” accessed on the internet at <http://en.wikipedia.org/wiki/Arabic—script—in—Unicode>; downloaded Dec. 22, 2014; 18pgs.Applicant
• “Chromium-compact-language-detector,” accessed on the internet at <https://code.googie.com/p/chromium-compact-language-detector/>; downloaded Dec. 22, 2014; 1 pg.Applicant
• “CJK Unified Ideographs (Unicode block),” accessed on the internet at <http://en.wikipedia.org/wiki/CJK—Unified—Ideographs—(Unicode block)>; downloaded Dec. 22, 2014; 1 pg.Applicant
• “CJK Unified Ideographs,” accessed on the internet at <http://en.wikipedia.org/wiki/CJK—Unified—Ideographs>, downloaded Dec. 22, 2014; 11 pgs.Applicant
• “cld2,” accessed on the internet at <https://code.google.com/p/cld2/>; downloaded Dec. 22, 2014; 2pgs.Applicant
• “Cyrillic script in Unicode,” accessed on the internet at <http://en.wikipedia.org/wiki/Cyrillic—script—in—Unicode>; downloaded Dec. 22, 2014; 22pgs.Applicant
• “Detect Method,” accessed on the internet at <http://msdn.microsoft.com/enus/library/ff512411.aspx>; downloaded Dec. 22, 2014; 5pgs.Applicant
• “GitHub,” accessed on the internet at <https://github.com/feedbackmine/language—detector>; downloaded Dec. 22, 2014; 1pg.Applicant
• “Google Translate API,” accessed on the internet at <https://cloud.qooqle.com/translate/v2/using—rest>; downloaded Dec. 22, 2014; 12pgs.Applicant
• “Idig (Language Detection with Infinity Gram),” accessed on the internet at <https://github.com/shuyo/Idig>; downloaded Dec. 22, 2014; 3pgs.Applicant
• “Language identification,” accessed on the internet at <http://en.wikipedia.org/wiki/Language—identification>; downloaded Dec. 22, 2014; 5pgs.Applicant
• “Languages and Scripts, CLDR Charts,” accessed on the internet at <http://www.unicode.org/cldr/charts/latest/ supplemental/lang uages—and—scripts.html>; downloaded Dec. 22, 2014; 23pgs.Applicant
• “Latin Script in Unicode,” accessed on the internet at <http://en.wikipedia.org/wiki/Latin—script—in—Unicode>; downloaded Dec. 22, 2014; 5pgs.Applicant
• “Mimer SQL Unicode Collation Charts,” accessed on the internet at <http://developer.mimer.com/charts/index. html>; downloaded Dec. 22, 2014; 2pgs.Applicant
• “Multi Core and Parallel Processing,” accessed on the internet at stackoverflow.com/questions/1922465/multi-core-and-parallel-processing, published Dec. 17, 2009; downloaded on Jun. 30, 2015; 2pgs.Applicant
• “Scripts and Languages,” accessed on the internet at <http://www.unicode.org/cldr/charts/latest/supplemental/scripts—and—languages.html>, downloaded Dec. 22, 2014; 23pgs.Applicant
• “Supported Script,” accessed on the internet at <http://www.unicode.org/standard/supported.html>; downloaded Dec. 22, 2014; 3pgs.Applicant
• “Unicode Character Ranges,” accessed on the internet at <http://jrgraphix.net/research/unicode—blocks.php>; downloaded Dec. 22, 2014; 1 pg.Applicant
• “Uscript.h File Reference,” accessed on the internet at <http://icuproject.org/apiref/icu4c/uscript—8h.html>, downloaded Dec. 22, 2014; 34pgs.Applicant