该软件专门针对论坛类采集.目前可以完成数据采集和文件下载功能,
软件的具体功能请见手册.目前还在测试期.
作者:小文 发布于:2010-11-25 23:16 Thursday 分类:开发计划
作者:小文 发布于:2010-11-17 21:49 Wednesday 分类:软件培训
示例网址:http://news.jyb.cn/zyjy/zyjyxw/201005/t20100512_359552.html
这个可以考虑使用插件,我们讲一下如何制作插件
作者:小文 发布于:2010-11-17 21:26 Wednesday 分类:软件培训
这个站,http://life.lygbst.cn/LifeComm/Colligation.aspx?AreaID=0 里边的详细内容是经过特殊处理的,我们需要使用插件将其还原.
作者:小文 发布于:2010-11-17 19:56 Wednesday 分类:软件培训
我们在中华英才网进行职位搜索时,会发现它的网址一直是变化的,比如 http://searchjob.chinahr.com/ ,我们从这个页面上输入关键字和其它条件时,就可以获取内容.
但我们要获取这个网址列表是不容易的,经分析,它的网址是经过urlencode,baseb6加密等方法进行处理的,直接是无法获取的.本次我们只讲一下如何分析这个网址.
需要的工具:
fiddler,firefox(组件firebug).
参考代码:
2
base64_decode
occIDList%3D604%252c600%26occParentIDList%3D604%252c600%26myLocIDList%3D30000%26myLocParentIDList%3D30000%26companyTypeIDList%3D%26isInterView%3D1%26from%3Dsearch%26IsModel%3Dfalse%26positionName%3D%25u9879%25u76ee%26prj%3Dwww1%26sj%3D1%26curPage%3D2%26pageSize%3D20%26recordCount%3D2000%26orderField%3D%26order%3DDESC
urldecode
occIDList=604%2c600&occParentIDList=604%2c600&myLocIDList=30000&myLocParentIDList=30000&companyTypeIDList=&isInterView=1&from=search&IsModel=false&positionName=%u9879%u76ee&prj=www1&sj=1&curPage=2&pageSize=20&recordCount=2000&orderField=&order=DESC
urldecode
occIDList=604,600&occParentIDList=604,600&myLocIDList=30000&myLocParentIDList=30000&companyTypeIDList=&isInterView=1&from=search&IsModel=false&positionName=项目&prj=www1&sj=1&curPage=2&pageSize=20&recordCount=2000&orderField=&order=DESC
1
occIDList%3D604%252c600%26occParentIDList%3D604%252c600%26myLocIDList%3D30000%26myLocParentIDList%3D30000%26companyTypeIDList%3D%26isInterView%3D1%26from%3Dsearch%26IsModel%3Dfalse%26positionName%3D%25u9879%25u76ee%26prj%3Dwww1%26sj%3D1%26curPage%3D1%26pageSize%3D20%26recordCount%3D2000%26orderField%3D%26order%3DDESC
occIDList=604%2c600&occParentIDList=604%2c600&myLocIDList=30000&myLocParentIDList=30000&companyTypeIDList=&isInterView=1&from=search&IsModel=false&positionName=%u9879%u76ee&prj=www1&sj=1&curPage=1&pageSize=20&recordCount=2000&orderField=&order=DESC
occIDList=604%2c600&occParentIDList=604%2c600&myLocIDList=30000&myLocParentIDList=30000&companyTypeIDList=&isInterView=1&from=search&IsModel=false&positionName=%e9%a1%b9%e7%9b%ae&prj=www1
occIDList%3d604%2c600%26occParentIDList%3d604%2c600%26myLocIDList%3d30000%26myLocParentIDList%3d30000%26companyTypeIDList%3d%26isInterView%3d1%26from%3dsearch%26IsModel%3dfalse%26positionName%3d%e9%a1%b9%e7%9b%ae%26prj%3dwww1%26curPage%3d1%26pageSize%3d20%26recordCount%3d2000%26orderField%3d%26order%3dDesc%26sj%3d1
occIDList=604,600&occParentIDList=604,600&myLocIDList=30000&myLocParentIDList=30000&companyTypeIDList=&isInterView=1&from=search&IsModel=false&positionName=项目&prj=www1&sj=1&curPage=1&pageSize=20&recordCount=2000&orderField=&order=DESC
作者:小文 发布于:2010-11-15 16:16 Monday 分类:软件培训
我们以 http://www.bookuu.com/search/book_search.jsp 为例,讲一下如何使用post采集.请大家先查看帮助手册中的 POST方式获得网址 一节内容.