基于条件随机场的非规范化中文地址解析方法
|
摘要
地址解析是地理编码的核心任务之一,而混乱的地址标准、随意的中文地址表达给地址解析带来了极大困难。该文提出一种基于条件随机场的非规范化中文地址解析方法。一方面,综合分析各种非规范化地址要素的类型特征,在现有标注体系基础上设计出一套优化的地址要素分类标注体系,并制定特征模板,然后采用自训练半监督学习与人工标注互补融合的策略,获取大量高质量的已标注语料供模型训练;另一方面,挑选已标注语料训练条件随机场模型,实现对地址要素的自动解析。选取广东省博罗县30 000条地址进行算法验证和解析性能评测。实验表明,与其他语料标注方法相比,该方法在获取有效地址解析的同时,显著降低了标注成本。结果表明,该方法适用于地理编码领域中大规模非规范化中文地址的自动解析。
Semantic resolution of Chinese addresses,as one of the core tasks of geocoding,splits unstructured Chinese addresses into elements,and at the same time identifies the types of those address elements.However,the chaotic standards and arbitrary expression of Chinese addresses have brought great difficulty and workload to address resolution.In this paper,a new approach to semantic resolution of Chinese text addresses based on the conditional random fields(CRF) and self-training techniques is propose.First of all,analysis of the structure and feature distribution on the large number of primitive address elements is performed via manual annotation and artificial correction,aiming at generating an optimized classification annotation system.Then,a semi-supervised self-training CRF based semantic resolution approach is proposed to effectively and efficiently annotate the large-scale corpus.The proposed method was evaluated by using 30 000 addresses from Boluo,Guangdong Province,China.The obtained result demonstrates that the proposed CRF based semantic resolution approach can obtain very promising precision rate,recall rate and F1 value,which indicate that the solution is suitable for automatic parsing of large-scale Chinese addresses in the field of geocoding.
Semantic resolution of Chinese addresses,as one of the core tasks of geocoding,splits unstructured Chinese addresses into elements,and at the same time identifies the types of those address elements.However,the chaotic standards and arbitrary expression of Chinese addresses have brought great difficulty and workload to address resolution.In this paper,a new approach to semantic resolution of Chinese text addresses based on the conditional random fields(CRF) and self-training techniques is propose.First of all,analysis of the structure and feature distribution on the large number of primitive address elements is performed via manual annotation and artificial correction,aiming at generating an optimized classification annotation system.Then,a semi-supervised self-training CRF based semantic resolution approach is proposed to effectively and efficiently annotate the large-scale corpus.The proposed method was evaluated by using 30 000 addresses from Boluo,Guangdong Province,China.The obtained result demonstrates that the proposed CRF based semantic resolution approach can obtain very promising precision rate,recall rate and F1 value,which indicate that the solution is suitable for automatic parsing of large-scale Chinese addresses in the field of geocoding.
引文
[1] 于滨,程昌秀,左廷英.面向全国经济普查需求的专家系统地理编码方法[J].计算机应用研究,2010,27(8):2976-2979.
[2] 于焕菊,李云岭,齐清文.顾及实体空间关系的地址编码方法研究[J].地理与地理信息科学,2013,29(5):49-52.
[3] 江洲,李琦.地理编码(Geocoding)的应用研究[J].地理与地理信息科学,2003,19(3):22-25.
[4] 张雪英,闾国年,李伯秋,等.基于规则的中文地址要素解析方法[J].地球信息科学学报,2010,12(1):9-16.
[5] 徐娟,曹晔,张奇.面向自由文本的中文地址规范化[J].计算机应用与软件,2015,32(8):22-24.
[6] 宋子辉.自然语言理解的中文地址匹配算法[J].遥感学报,2013,17(4):788-801.
[7] 吴海涛,俞立,张贵军.基于模糊匹配策略的城市中文地址编码系统[J].计算机工程,2011,37(2):194-196.
[8] 马照亭,李志刚,孙伟,等.一种基于地址分词的自动地理编码算法[J].测绘通报,2011(2):59-62.
[9] 程昌秀,于滨.一种基于规则的模糊中文地址分词匹配方法[J].地理与地理信息科学,2011,27(3):26-29.
[10] 罗明,黄海量.一种基于有限状态机的中文地址标准化方法[J].计算机应用研究,2016,33(10):3691-3695.
[11] 亢孟军,杜清运,王明军.地址树模型的中文地址提取方法[J].测绘学报,2015,44(1):99-107.
[12] 赵卫锋,张勤.非结构化中文自然语言地址描述的自动识别[J].计算机工程与应用,2016,52(23):19-24.
[13] 臧英斐.基于语义分析的地址匹配研究[D].重庆:重庆交通大学,2015.
[14] 梁东阳.中文地址名称识别算法设计和实现[D].天津:天津大学,2015.
[15] 王克永,刘纪平,罗安,等.前后缀与特征词相结合的地名地址提取[J].测绘通报,2016(2):64-68.
[16] 李晓林,黄爽,卢涛,等.非规范化中文地址的行政区划提取算法[J].计算机应用,2017,37(3):876-882.
[17] 王勇,刘纪平,郭庆胜,等.顾及位置关系的网络POI地址信息标准化处理方法[J].测绘学报,2016,45(5):623-630.
[18] 蒋文明,张雪英,李伯秋.基于条件随机场的中文地址要素识别方法[J].计算机工程与应用,2010,46(13):129-131.
[19] 周海.基于条件随机场和空间推理的地理编码方法[D].郑州:解放军信息工程大学,2015.
[20] SUTTON C,MCCALLUM A.An introduction to conditional random fields[J].Foundations and Trends○R in Machine Learning,2012,4(4):267-373.
[21] LAFFERTY J D,MCCALLUM A,PEREIRA F C N.Conditional random fields:Probabilistic models for segmenting and labeling sequence data[A].Eighteenth International Conference on Machine Learning[C].2001.282-289.
[22] 何炎祥,罗楚威,胡彬尧.基于CRF和规则相结合的地理命名实体识别方法[J].计算机应用与软件,2015,32(1):179-185.
[23] 邬伦,刘磊,李浩然,等.基于条件随机场的中文地名识别方法[J].武汉大学学报(信息科学版),2017,42(2):150-156.
[24] POULIQUEN B,KIMLER M,STEINBERGER R,et al.Geocoding multilingual texts:Recognition,disambiguation and visualisation[A].International Conference on Linguistic Resources and Evaluation[C].2006.1-2.
[25] LI L,DING Z,HUANG D.Recognizing location names from Chinese texts based on Max-Margin Markov Network[A].IEEE International Conference on Natural Language Processing and Knowledge Engineering[C].2008.1-7.
[26] 刘孝良.基于半监督学习的随机森林算法研究与应用[D].青岛:中国海洋大学,2013.
[27] 宗成庆.统计自然语言处理[M].北京:清华大学出版社,2013.
[2] 于焕菊,李云岭,齐清文.顾及实体空间关系的地址编码方法研究[J].地理与地理信息科学,2013,29(5):49-52.
[3] 江洲,李琦.地理编码(Geocoding)的应用研究[J].地理与地理信息科学,2003,19(3):22-25.
[4] 张雪英,闾国年,李伯秋,等.基于规则的中文地址要素解析方法[J].地球信息科学学报,2010,12(1):9-16.
[5] 徐娟,曹晔,张奇.面向自由文本的中文地址规范化[J].计算机应用与软件,2015,32(8):22-24.
[6] 宋子辉.自然语言理解的中文地址匹配算法[J].遥感学报,2013,17(4):788-801.
[7] 吴海涛,俞立,张贵军.基于模糊匹配策略的城市中文地址编码系统[J].计算机工程,2011,37(2):194-196.
[8] 马照亭,李志刚,孙伟,等.一种基于地址分词的自动地理编码算法[J].测绘通报,2011(2):59-62.
[9] 程昌秀,于滨.一种基于规则的模糊中文地址分词匹配方法[J].地理与地理信息科学,2011,27(3):26-29.
[10] 罗明,黄海量.一种基于有限状态机的中文地址标准化方法[J].计算机应用研究,2016,33(10):3691-3695.
[11] 亢孟军,杜清运,王明军.地址树模型的中文地址提取方法[J].测绘学报,2015,44(1):99-107.
[12] 赵卫锋,张勤.非结构化中文自然语言地址描述的自动识别[J].计算机工程与应用,2016,52(23):19-24.
[13] 臧英斐.基于语义分析的地址匹配研究[D].重庆:重庆交通大学,2015.
[14] 梁东阳.中文地址名称识别算法设计和实现[D].天津:天津大学,2015.
[15] 王克永,刘纪平,罗安,等.前后缀与特征词相结合的地名地址提取[J].测绘通报,2016(2):64-68.
[16] 李晓林,黄爽,卢涛,等.非规范化中文地址的行政区划提取算法[J].计算机应用,2017,37(3):876-882.
[17] 王勇,刘纪平,郭庆胜,等.顾及位置关系的网络POI地址信息标准化处理方法[J].测绘学报,2016,45(5):623-630.
[18] 蒋文明,张雪英,李伯秋.基于条件随机场的中文地址要素识别方法[J].计算机工程与应用,2010,46(13):129-131.
[19] 周海.基于条件随机场和空间推理的地理编码方法[D].郑州:解放军信息工程大学,2015.
[20] SUTTON C,MCCALLUM A.An introduction to conditional random fields[J].Foundations and Trends○R in Machine Learning,2012,4(4):267-373.
[21] LAFFERTY J D,MCCALLUM A,PEREIRA F C N.Conditional random fields:Probabilistic models for segmenting and labeling sequence data[A].Eighteenth International Conference on Machine Learning[C].2001.282-289.
[22] 何炎祥,罗楚威,胡彬尧.基于CRF和规则相结合的地理命名实体识别方法[J].计算机应用与软件,2015,32(1):179-185.
[23] 邬伦,刘磊,李浩然,等.基于条件随机场的中文地名识别方法[J].武汉大学学报(信息科学版),2017,42(2):150-156.
[24] POULIQUEN B,KIMLER M,STEINBERGER R,et al.Geocoding multilingual texts:Recognition,disambiguation and visualisation[A].International Conference on Linguistic Resources and Evaluation[C].2006.1-2.
[25] LI L,DING Z,HUANG D.Recognizing location names from Chinese texts based on Max-Margin Markov Network[A].IEEE International Conference on Natural Language Processing and Knowledge Engineering[C].2008.1-7.
[26] 刘孝良.基于半监督学习的随机森林算法研究与应用[D].青岛:中国海洋大学,2013.
[27] 宗成庆.统计自然语言处理[M].北京:清华大学出版社,2013.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |