Python-Node Data Automation Scripts
$25-50 USD / hour
Posted 1 minute ago
•
Ends in 6 days
$25-50 USD / hour
This project has three connected pieces and I’d like to tackle them in the order that keeps the workflow smooth for everyone. Phase 1 – Website-to-Excel Scraper We begin by coding a Python scraper that pulls data directly from a set of websites and drops it into clean, well-structured Excel workbooks. The scraper should cope with pagination, basic authentication when needed, and changing HTML layouts (lxml or BeautifulSoup are both fine). I will provide the target URLs and the exact column layout; you make the script resilient and easy to re-point when sites change. As soon as the data lands in Excel, a short Pandas routine should run simple sanity checks so I can trust the numbers. Phase 2 – Automated Word Report Formatting Once the data is in place, a second Python script should transform the analysis output into a polished Word report. Using python-docx and lxml, the script must: • switch every section to landscape orientation, • preserve complex table structures, and • embed any charts or summary text I pass in. The goal is a single-click command that turns raw numbers into a board-ready .docx file without manual tweaking. Phase 3 – Node.js Integration Layer Finally, I need a lightweight Node.js app (Express is fine) where a user can upload new configuration files or even raw data, hit “Run Collection,” and watch the scraper + formatter pipeline execute in the background. Returned status, logs, and finished reports should be delivered as downloadable links and mirrored in a simple JSON API so other services can hook in later. Acceptance check • Scraper writes fully populated Excel files with zero empty header cells. • Formatter outputs a landscape Word report whose style matches the sample template I’ll provide. • Web app uploads a file <20 MB, triggers the Python scripts, and responds with success/fail plus a link to the result within 60 seconds. If you’re comfortable jumping between Python (requests, Pandas, python-docx, lxml) and Node.js (Express, multer, child_process), let’s get started.
Project ID: 40812148
About the project
36 proposals
Open for bidding
Remote project
Active 3 secs ago
Place your bid
Bid amount Email addressBenefits of bidding on Freelancer
Set your budget and timeframe
It's free to sign up and bid on jobs
36 freelancers are bidding on average $32 USD/hour for this job


Hi, This is Elias from Miami. I checked your project description and understand you're looking to create Python and Node.js automation scripts that are interconnected. It’s essential to tackle these pieces in a logical order to ensure smooth execution. I have experience building similar automation systems and understand the key challenges involved. I typically approach projects by first mapping out the workflow and identifying how each component interacts. I have a few questions to get a better understanding: Q1 – What specific tasks should each of the three scripts handle? Q2 – Are there any existing systems or APIs we need to integrate with? Q3 – What kind of data outputs or reports do you expect from the automation? Looking forward to hearing from you.


Hello, The three phases fit together well, and I would approach them as one coordinated pipeline rather than three separate scripts. This will help keep the data flow consistent and make future changes easier. I have experience with Python, web scraping, Pandas, Excel automation, and document processing. I can build the scraper with configurable URLs and column mappings, handle pagination and authentication, and add validation checks before exporting the results to Excel. For the Word stage, I would use python-docx and lxml to apply consistent landscape formatting, preserve table structures, and insert the supplied charts and summary content while matching your sample template. The Node.js layer can then provide file uploads, background execution of the Python scripts, progress/status reporting, logs, and downloadable results through both the web interface and a JSON API. I would keep the components modular, with clear error handling between stages. I would also verify the 60-second response requirement against the actual processing workload, since larger collections may need asynchronous job handling. Once the target websites, sample Excel layout, Word template, and configuration format are available, I can review the scope and suggest the implementation structure. Deliverables will include the Python scripts, Node.js application, and instructions for configuration and running the complete workflow.


Your "Run Collection" button will start the scraper and the python-docx formatter as one tracked job. Each site's selectors live in a config file, so when a site changes its layout you edit one line rather than rewrite the scraper. What makes or breaks this project is how Node hands the job to Python. Express starts the scripts with child_process, and every run writes its status and log to a job record that your JSON API reads. If any header cell comes back empty, the job fails before bad data reaches the Word report. Honestly, the hard part is keeping complex tables intact in python-docx, so we'll build against your sample template from the start. The first delivery is the Excel scraper with its Pandas checks. The formatter and the web app come after. We built a Python scraping bot for a client here that sends live alerts about warehouse jobs in Canada, 5★: https://www.freelancer.com/projects/web-scraping/Build-Real-Time-Canada-Warehouse Do any of the target sites need a login, or are they all public?


Hello, I can help you with "Python-Node Data Automation Scripts" as per your given set of instructions. Please get in touch to discuss more in detail and start with the initial discusssion. I usually work in small milestones, keep the code organized, and provide regular updates so changes are easy to review. Let's discuss the current setup. I'll suggest the best implementation and an accurate timeline. ⭐ 5.0/5 from a recent client: "Excellent work. Would definitely utilize again. Stellar job!" Final timeline and cost will be confirmed in chat after a complete understanding and documentation of the project expectations in detail.


Hello!, This is James from Hollywood... I read your post carefully, and the main pain point here is not just “writing scripts” but making sure the 3 connected pieces work cleanly together, in the right order, without creating messy handoff problems later. I can help you break it into a simple, stable flow: first map the data sources, then build the Python/Node automation layer, then validate the output in Excel/CSV so it’s reliable and easy to rerun. I’ve built automation pipelines with Python, Node.js, Pandas, BeautifulSoup, and Excel-based reporting before, and I’m very comfortable with web scraping and software architecture decisions that keep things maintainable instead of fragile. My goal would be to deliver something you can actually reuse, not just a one-off script. A few relevant examples: - lead enrichment scraper + CSV export workflow - inventory sync automation for an e-commerce dashboard - Python/Node reporting pipeline for finance ops - data collection script with Excel output and cleanup rules Quick questions: 1) What are the 3 connected pieces exactly, and which one should be tackled first? 2) What are the input and output formats you want for each step? 3) Are there any target websites or rate-limit/captcha concerns I should plan around? If you want, I can start by outlining the safest implementation order before coding, so we avoid rework and keep this within budget.


I can help you build this three-phase pipeline to streamline your workflow from data collection to report generation. For Phase 1, I will develop a resilient Python scraper using requests and BeautifulSoup, complete with pagination and basic authentication handling. I will integrate Pandas to run immediate sanity checks on the data, ensuring clean Excel workbooks with fully populated header cells. For Phase 2, I will write a Python script utilizing python-docx and lxml to programmatically transform your analysis output into a landscape Word report that preserves complex tables and embeds your charts seamlessly. For Phase 3, I will build a lightweight Express application in Node.js to handle file uploads under 20 milliseconds, securely trigger the Python scripts in the background, and return the requested status, logs, and downloadable reports within your 60-second window alongside a JSON API. Let us discuss the target URLs and your sample template to get started.


Hello, Handling this end-to-end pipeline requires a clean handoff between the Python extraction scripts and the Node.js orchestration layer. For the scraper in Phase 1, using BeautifulSoup alongside Pandas ensures fast parsing while catching schema drifts before the data hits your Excel workbooks. Phase 2 relies on python-docx to handle the landscape orientation switches and complex table styling without breaking document integrity. Finally, wrapping these scripts inside an Express application using child processes will keep the execution asynchronous and safely bounded by your performance targets. Are the target websites requiring session cookies through basic authentication, or will standard header management suffice for the scraper? Have a nice day


Hello, Looking at your three phases, the cleanest path is to lock the Python scraper and Word formatter as reliable command-line tools first, then wrap them with Node once both scripts behave the same way every run. That way Excel quality and the landscape report are proven before any upload UI is in the mix. For Phase 1 I would build a URL-driven scraper with clear selectors and retries so pagination and light auth still work when a site tweaks its HTML. After the pull, a short Pandas pass would flag empty headers, odd blanks, and type quirks so you can trust the workbook before anything else runs. Phase 2 would treat your sample template as the style source: landscape sections, tables kept intact, charts and summary text dropped in with python-docx so one command turns the numbers into a board-ready docx. Phase 3 is a light Express app with multer for uploads under 20 MB, then child_process to kick off the Python pipeline in the background. Status, logs, and download links show up in the UI and as simple JSON so other services can hook in later. I can put in 40 hours each week and aim to ship a working end-to-end flow in about a week once I have your URLs and template. Happy to start with the first scrape target so we lock the column layout early.


Your three-phase workflow is a strong fit for a clean, maintainable automation pipeline. I can build the Python scraper with pagination, authentication, configurable selectors, resilient parsing, Excel generation, and Pandas validation checks, including safeguards for missing headers and incomplete records. The reporting layer can transform those outputs into a polished landscape Word document while preserving complex tables, incorporating charts and summary content, and following your sample template closely. I can then connect everything through a lightweight Express API with secure uploads, background execution, progress/status tracking, logs, downloadable results, and integration-friendly JSON responses. I’ll structure the solution so website mappings, column definitions, report settings, and runtime options are easy to update without rewriting the core pipeline. The <20 MB upload limit and 60-second response requirement will be addressed with validation, process control, and clear success/failure handling. The result will be a practical single-click workflow that is easier to operate, troubleshoot, and extend as your data sources evolve. Best regards


I understand you need a robust Python scraper to extract data from websites, mirroring the need for dynamic parsing demonstrated in similar automation projects I've completed. My experience with `BeautifulSoup` and `lxml` has equipped me to handle pagination and evolving HTML structures efficiently, ensuring data integrity even with minor site changes. My approach will involve developing a modular Python script utilizing `requests` for HTTP interactions and `BeautifulSoup` for parsing HTML. I'll implement robust error handling and logging to monitor script execution and identify any issues promptly. For pagination, I'll employ iterative URL generation based on common patterns or explicit next-page links, and authentication will be handled via session management. The output will be structured Pandas DataFrames, then exported to CSV or Excel as specified. To ensure seamless integration, how will the provided target URLs be structured, and are there any specific authentication credentials I should anticipate? I'm ready to discuss the technical details further and outline a precise implementation plan.


⚠️ If you're not happy, you don’t pay. ⚠️ Hi Client, thank you for checking my proposal and sharing the detailed project brief. I can build your data processing pipeline using Python and Node.js with a robust and scalable design. I will deliver: - A resilient Python scraper that handles pagination and authentication, ensuring clean Excel outputs. - Pandas integration for sanity checks on incoming data. - A Python script using python-docx to produce polished, landscape-oriented Word reports with complex tables and embedded charts. - A lightweight Node.js app (Express) for seamless user uploads and real-time execution of the scraper and formatter. - A JSON API for easy integration with other services and status updates on the process. You will also receive detailed documentation and support for future modifications. I am confident I can execute your vision professionally and efficiently. Looking forward to discussing timeline and next steps. Best regards, Chirag Pipal


the scraper, the landscape word report and the run collection button all sound very doable, and zero empty header cells is easy to hold to i can build all three pieces for you, with pagination and login handled i'll stay on it until your three checks pass, with bug fixes and support after handover do most of the target sites need a login or only a few? send the urls and template when you're ready Sela
内容来自 Freelancer News,版权归原作者所有。
阅读原文 ↗