基于多线程网络爬虫技术的气象雷达数据采集系统

论文总字数：33275字

目 录

1 引言...................................................1

1.1 研究背景及意义 1

1.2 本课题研究内容与目标 1

1.3 论文内容的组织 2

2 相关技术...............................................2

2.1 网络爬虫技术 2

2.1.1 网络爬虫的基本原理及工作流程 2

2.1.2 网页抓取策略 3

2.1.3 网络爬虫分类 4

2.1.4 网络爬虫的工作方式 6

2.2 Web信息提取 6

2.3 java多线程 7

2.3.1 线程的基本概念 7

2.3.2 多线程机制 8

2.3.3 Windows下的多线程技术 10

2.4 JSP（java服务器页面） 11

2.4.1 JSP简介 11

2.4.2 JSP的优点 11

2.5 MVC框架 11

2.5.1 JSP JavaBean 模式 12

2.5.2 MVC 模式简介 12

2.5.3 MVC使用原因 14

2.6天气雷达技术 15

2.6.1 天气雷达原理 15

2.6.2 天气雷达特点 15

2.6.3 天气雷达发展简史..............................................................................................16

2.7本章小结............................................................................................................................17

3 系统设计与实现..........................................17

3.1系统层次结构设计............................................................................................................17

3.2 系统实现...........................................................................................................................18

3.2.1 表现层....................................................................................................................18

3.2.2 业务逻辑层............................................................................................................24

3.2.3　数据访问层..........................................................................................................25

3.3 本章小结.........................................................................................................................26

4 系统测试................................................26

4.1系统测试的目标................................................................................................................26

4.2 测试的实现.......................................................................................................................27

5 总结与展望..............................................28

5.1 总结...................................................................................................................................28

5.2 展望...................................................................................................................................28

参考文献 ..................................................28

致谢......................................................30

基于多线程网络爬虫技术的气象雷达数据采集系统

杨求龙

,China

Abstract:With the computer and network technology development is getting mature day after day, the computer technology and network technology is applied to various fields, the number of network information is getting larger and larger at a very impressive speed, and thus produced a large amount of data, which makes people on the Internet to find the information they really need is more and more difficult.In this paper,I will introduce a very important instrument in collecting web pages- web crawler.The main function of Web crawler is to collect web pages, you can use it to collect a web site content regularly, or to track the development of the web site, or as the intra-site search engine.The name-"Web Crawler" is very vivid, it is in the Internet, via the web link (URL), from the current page climbed to a next web page to collect content.The task of a Web crawler is to obtain the title and summary of the page ;put the collected page title, link, summaries of web pages into the database; according to the content of the current page to collect page link information and according to the link combined with the corresponding algorithm and sequentially search page content corresponded to the link.

Traditional search engine's goal is to reach as high as possible in the network coverage rate and may provide comprehensive information, but high network coverage rate will lead to the information it provides to the user contains too much useless information, which increase the user of the difficulty in obtaining useful information.

　　In addition, the traditional search engine for specific areas of information retrieval results are not professional enough, can not meet the specific needs of specific areas, specific professional groups.

剩余内容已隐藏，请支付后下载全文，论文总字数：33275字

相关图片展示：