Web Crawling Jobs

Join our growing health insurance sales operation as a Lead Generation & Web Scraping Specialist. We are looking for a highly skilled resourceful and results-driven professional who can locate collect clean and organize large volumes of targeted lead data from publicly available online sources. This is not a basic data-entry position. We are specifically looking for someone with strong experience in web scraping data mining lead generation automation and large-scale data management who can consistently produce high-quality usable lead lists. Our primary target market includes self-employed individuals business owners independent contractors and other high-value prospects who may be a strong fit for private health insurance options. Your primary responsibilities will include: Researching and identifying valuable public directories licensing databases business listings association websites government databases and other lead sources. Scraping and collecting publicly available information such as business names owner names phone numbers locations websites and other relevant contact data. Finding sources that contain direct mobile phone numbers when publicly available. Discovering new and valuable lead sources independently rather than relying only on websites provided by the company. Building and maintaining efficient scraping systems capable of handling large volumes of data. Cleaning formatting standardizing and organizing large datasets. Removing duplicate records and identifying incomplete outdated or low-quality leads. Validating data quality and improving the accuracy of phone numbers and contact information where possible. Organizing leads by state industry business type profession or other targeting criteria. Managing large CSV files spreadsheets and lead databases efficiently. Keeping clear records of the original source for each dataset. Troubleshooting difficult websites changing page structures access limitations and other common scraping challenges. Continuously improving scraping processes automation speed data quality and overall lead production. Production Expectations This is a high-volume lead generation position. Our goal is to consistently generate approximately 60 000-100 000 usable leads per week once the workflow and systems are fully established. We are not interested in receiving 100 000 random scraped rows. We are looking for clean organized relevant and usable data that meets the targeting requirements provided. Quality is just as important as volume. Strong candidates should be capable of building scalable scraping processes rather than relying entirely on manual research. Required Skills and Experience Strong experience with web scraping and data extraction. Experience generating large targeted lead lists. Ability to identify and research valuable data sources independently. Strong data cleaning deduplication and organization skills. Experience working with large CSV files and datasets. Strong attention to detail and quality control. Ability to troubleshoot scraping issues independently. Comfortable working with Google Sheets and Microsoft Excel. Highly organized and able to manage multiple data sources and projects. Reliable self-motivated and capable of working independently. Highly Preferred Technical Skills Experience with Python and scraping libraries. Browser automation tools such as Selenium or Playwright. Scraping platforms and automation software. API data extraction. Proxy and IP rotation management. Responsible rate limiting and scraper monitoring. Data enrichment and phone validation tools. Regular expressions and data parsing. Experience working with public records licensing databases government websites business directories and association databases. Experience building reusable or automated scraping workflows is a major advantage. The Ideal Candidate The ideal person for this position is not simply someone who knows how to scrape a website. We are looking for someone who is creative analytical persistent and highly resourceful. You should enjoy researching new sources of data and finding opportunities that other lead generators may overlook. You should be comfortable receiving a target market and independently determining where the best data can be found. We highly value candidates who can think strategically about lead generation and continuously improve the quality and volume of the data being produced. Position and Growth Opportunity This position is intended to develop into a long-term full-time opportunity for the right person. We expect approximately 35-40 hours per week as the role develops although performance and results are more important than simply tracking hours. Compensation and responsibilities can increase significantly for someone who consistently demonstrates the ability to: Produce high volumes of usable leads. Maintain strong data quality. Discover valuable new lead sources. Improve and automate scraping processes. Work independently without constant supervision. Consistently meet production goals. We are looking for someone who wants to become a key part of a growing sales operation rather than simply complete occasional scraping projects. If you have proven experience in web scraping data mining automation lead generation and building large targeted datasets we would love to hear from you. When applying please include examples of previous scraping projects the approximate number of records you have generated the tools or programming languages you use and any experience you have working with large-scale lead generation.