Negative sampling improves hypernymy extraction based on projection learning / Ustalov D., Arefyev N., Biemann C., Panchenko A. // 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017 - Proceedings of Conference. - 2017. - V. 2, l. . - P. 543-550.

ISSN:
нет данных
Type:
Conference Paper
Abstract:
We present a new approach to extraction of hypernyms based on projection learning and word embeddings. In contrast to classification-based approaches, projection-based methods require no candidate hyponym-hypernym pairs. While it is natural to use both positive and negative training examples in supervised relation extraction, the impact of negative examples on hypernym prediction was not studied so far. In this paper, we show that explicit negative examples used for regularization of the model significantly improve performance compared to the stateof- the-art approach of Fu et al. (2014) on three datasets from different languages. © 2017 Association for Computational Linguistics.
Author keywords:
Index keywords:
Aluminum alloys; Classification (of information); Computational linguistics; Extraction; Linguistics; Embeddings; Hypernymy extraction; Improve performance; Negative examples; New approaches; Relation
DOI:
нет данных
Смотреть в Scopus:
https://www.scopus.com/inward/record.uri?eid=2-s2.0-85021627162&partnerID=40&md5=e2643064cbb13962caafac9251243822
Соавторы в МНС:
Другие поля
Поле Значение
Link https://www.scopus.com/inward/record.uri?eid=2-s2.0-85021627162&partnerID=40&md5=e2643064cbb13962caafac9251243822
Affiliations Ural Federal University, Institute of Natural Sciences and Mathematics, Russian Federation; Moscow State University, Faculty of Computational Mathematics and Cybernetics, Russian Federation; University of Hamburg, Deptartment of Informatics, Language Technology Group, Germany
References Abadi, M., TensorFlow: Large-scale machine learning on heterogeneous distributed systems (2016) CoRR, , abs/1603.04467; Baroni, M., Lenci, A., How we BLESSed distributional semantic evaluation (2011) Proceedings of the GEMS 2011 Workshop on GEometrical Models of Natural Language Semantics, GEMS '11, pp. 1-10. , Edinburgh, Scotland. Association for Computational Linguistics; Ferraresi, A., Zanchetta, E., Baroni, M., Bernardini, S., Introducing and evaluating ukWaC, a very large Web-derived corpus of English (2008) Proceedings of the 4th Web As Corpus Workshop (WAC-4): Can We Beat Google?, pp. 47-54. , Marakech, Morocco; Frome, A., Corrado, G.S., Shlens, J., Bengio, S., Dean, J., Aurelio Ranzato, M., Mikolov, T., DeViSE: A deep visual-semantic embedding model (2013) Advances in Neural Information Processing Systems, 26, pp. 2121-2129. , Curran Associates, Inc., Harrahs and Harveys, NV, USA; Fu, R., Guo, J., Qin, B., Che, W., Wang, H., Liu, T., Learning semantic hierarchies via word embeddings (2014) Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1199-1209. , Baltimore, MD, USA. Association for Computational Linguistics; Goldhahn, D., Eckart, T., Quasthoff, U., Building large monolingual dictionaries at the leipzig corpora collection: From 100 to 200 languages (2012) Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC'12), pp. 759-765. , Istanbul, Turkey. European Language Resources Association (ELRA); Gong, Z., Wa Cheang, C., Leong Hou, U., Web query expansion by wordnet (2005) Proceedings of the 16th International Conference on Database and Expert Systems Applications-DEXA '05, pp. 166-175. , Springer Berlin Heidelberg, Copenhagen, Denmark; Graff, D., (2003) English Gigaword, , Technical Report LDC2003T05, Linguistic Data Consortium, Philadelphia, PA, USA; Marti, A.H., Automatic acquisition of hyponyms from large text corpora (1992) Proceedings of the 14th Conference on Computational Linguistics-Volume 2, COLING'92, pp. 539-545. , Nantes, France. Association for Computational Linguistics; Heylen, K., Peirsman, Y., Geeraerts, D., Speelman, D., Modelling word similarity: An evaluation of automatic synonymy extraction algorithms (2008) Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08), pp. 3243-3249. , Marrakech, Morocco. European Language Resources Association (ELRA); Hochreiter, S., Schmidhuber, J., Long short-term memory (1997) Neural Computation, 9 (8), pp. 1735-1780; Kingma, D.P., Ba, J., Adam: A method for stochastic optimization (2014) CoRR, , abs/1412.6980; Krizhanovsky, A.A., Smirnov, A.V., An approach to automated construction of a general-purpose lexical ontology based on Wiktionary (2013) Journal of Computer and Systems Sciences International, 52 (2), pp. 215-225; Lenci, A., Benotto, G., Identifying hypernyms in distributional semantic spaces (2012) Proceedings of the First Joint Conference on Lexical and Computational Semantics-Volume 1: Proceedings of the Main Conference and the Shared Task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation, SemEval '12, pp. 75-79. , Montreal, Canada. Association for Computational Linguistics; Levy, O., Remus, S., Biemann, C., Dagan, I., Do supervised distributional methods really learn lexical inference relations? (2015) Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 970-976. , Denver, Colorado, USA. Association for Computational Linguistics; MacQueen, J., Some methods for classification and analysis of multivariate observations (1967) Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, 1, pp. 281-297. , Statistics, Berkeley, California, USA. University of California Press; Mikolov, T., Le, Q.V., Sutskever, I., Exploiting similarities among languages for machine translation (2013) CoRR, , abs/1309.4168; Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J., Distributed representations of words and phrases and their compositionality (2013) Advances in Neural Information Processing Systems, 26, pp. 3111-3119. , Curran Associates, Inc., Harrahs and Harveys, NV, USA; George, A.M., WordNet: A lexical database for english (1995) Communications of the ACM, 38 (11), pp. 39-41; Navigli, R., Velardi, P., Learning word-class lattices for definition and hypernym extraction (2010) Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pp. 1318-1327. , Uppsala, Sweden. Association for Computational Linguistics; Nayak, N., (2015) Learning Hypernymy over Word Embeddings, , Technical report, Stanford University; Necsulescu, S., Mendes, S., Jurgens, D., Bel, N., Navigli, R., Reading between the lines: Overcoming data sparsity for accurate classification of lexical relationships (2015) Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics, pp. 182-192. , Denver, CO, USA. Association for Computational Linguistics; Panchenko, A., Morozova, O., Naets, H., A semantic similarity measure based on lexico-syntactic patterns (2012) Proceedings of KONVENS 2012, pp. 174-178. , Vienna, Austria. OGAI; Panchenko, A., Faralli, S., Ruppert, E., Remus, S., Naets, H., Fairon, C., Paolo Ponzetto, S., Biemann, C., TAXI at semeval-2016 task 13: A taxonomy induction method based on lexico-syntactic patterns, substrings and focused crawling (2016) Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 1320-1327. , San Diego, CA, USA. Association for Computational Linguistics; Panchenko, A., Ustalov, D., Arefyev, N., Paperno, D., Konstantinova, N., Loukachevitch, N., Biemann, C., Human and machine judgements for Russian semantic relatedness (2016) Proceedings of the 5th Conference on Analysis of Images, Social Networks and Texts (AIST'2016), Volume 661 of Communications in Computer and Information Science, pp. 303-317. , Yekaterinburg, Russia. Springer-Verlag Berlin Heidelberg; Panchenko, A., Comparison of the baseline knowledge-, corpus-, and web-based similarity measures for semantic relations extraction (2011) Proceedings of the GEMS 2011 Workshop on GEometrical Models of Natural Language Semantics, pp. 11-21. , Edinburgh, UK. Association for Computational Linguistics; Roller, S., Erk, K., Boleda, G., Inclusive yet Selective: Supervised Distributional Hypernymy Detection (2014) Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pp. 1025-1036. , Dublin, Ireland, August. Dublin City University and Association for Computational Linguistics; Santus, E., Yung, F., Lenci, A., Huang, C.-R., EVALution 1 0 an evolving semantic dataset for training and evaluation of distributional semantic models (2015) Proceedings of the 4th Workshop on Linked Data in Linguistics: Resources and Applications, pp. 64-69. , Beijing, China. Association for Computational Linguistics; Santus, E., Lenci, A., Chiu, T.-S., Lu, Q., Huang, C.-R., Nine features in a random forest to learn taxonomical semantic relations (2016) Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), pp. 4557-4564. , Portoroz, Slovenia. European Language Resources Association (ELRA); Shwartz, V., Goldberg, Y., Dagan, I., Improving hypernymy detection with an integrated path-based and distributional method (2016) Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2389-2398. , Berlin, Germany. Association for Computational Linguistics; Snow, R., Jurafsky, D., Andrew, Y.N., Learning syntactic patterns for automatic hypernym discovery (2004) Proceedings of the 17th International Conference on Neural Information Processing Systems, NIPS'04, pp. 1297-1304. , Vancouver, British Columbia, Canada. MIT Press; Snow, R., Jurafsky, D., Andrew, Y.N., Semantic taxonomy induction from heterogenous evidence (2006) Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pp. 801-808. , Sydney, Australia. Association for Computational Linguistics; Tjong Kim Sang, E., Hofmann, K., Lexical patterns or dependency patterns: Which is better for hypernym extraction? (2009) Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL-2009), pp. 174-182. , Boulder, Colorado, USA. Association for Computational Linguistics; Vulic, I., Korhonen, A., On the role of seed lexicons in learning bilingual word embeddings (2016) Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 247-257. , Berlin, Germany. Association for Computational Linguistics; Vylomova, E., Rimell, L., Cohn, T., Baldwin, T., Take and took, gaggle and goose, book and read: Evaluating the utility of vector differences for lexical relation learning (2016) Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1671-1682. , Berlin, Germany. Association for Computational Linguistics; Wandmacher, T., How semantic is latent semantic analysis? (2005) Proceedings of RECITAL 2005, pp. 525-534. , Dourdan, France; Weeds, J., Clarke, D., Reffin, J., Weir, D., Keller, B., Learning to distinguish hypernyms and co-hyponyms (2014) Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pp. 2249-2259. , Dublin, Ireland. Dublin City University and Association for Computational Linguistics; Yamane, J., Takatani, T., Yamada, H., Miwa, M., Sasaki, Y., Distributional hypernym generation by jointly learning clusters and projections (2016) Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp. 1871-1879. , Osaka, Japan, December. The COLING 2016 Organizing Committee; Zhou, G., Liu, Y., Liu, F., Zeng, D., Zhao, J., Improving question retrieval in community question answering using world knowledge (2013) Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, IJCAI '13, pp. 2239-2245. , Beijing, China. AAAI Press
Sponsors CELI: Language Technology;eBay;et al.;Grammarly;Textkernel;Thomson Reuters
Publisher Association for Computational Linguistics (ACL)
Conference name 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017
Conference date 3 April 2017 through 7 April 2017
Conference code 127985
ISBN 9781510838604
Language of Original Document English
Abbreviated Source Title Conf. Eur. Chapter Assoc. Comput. Linguist., EACL - Proc. Conf.
Source Scopus