I want someone to generate a scraper which will scrape this website to get all of the database details, e.g. all the table names and fields under each table etc.. All information from each page.
The application should be able to check if the tables have changes and be updated based on a interval we set.
Be able to create a visual of all tables and selected tables of how they are connected, and be able to update if incorrect.
See a table view of the tables.
Be able to click a click/drag and drop for selected tables to create a sql query based on this. (example will be shared in chat). And save this query created.

I am looking for the backend to be built using python e.g. selenium/BeautifulSoup with RAG and the front end to be built using PHP frameworks such as Laravel, composer.
The database will be Mysql e.g. MariaDB
The front end UI should be able to search and filter tables, fields and other information stored in the DB
A login page to login to the application you are building



I would like to hear your suggestions on the best technology you would recommend to use.

Website to scrape example: https://docs.oracle.com/en/cloud/saas/human-resources/24a/oedmh/perassignmentsv-6982.html#perassignmentsv-6982

Budget: $50
Posted On: May 04, 2024 11:33 UTC
Category: Full Stack Development
Skills:PHP, Python, JavaScript, MySQL Programming, Data Scraping, jQuery
Skills: PHP, Python, JavaScript, MySQL Programming, Data Scraping, jQuery
Country: United Kingdom
click to apply

Cover Letter

Hi there! I’m David, a seasoned Laravel developer with a strong background in PHP, having over 10 years of extensive experience. Since Laravel 4.2, I’ve been immersed in everything Laravel-related, including working with vue.js, TailwindCSS, Bootstrap CSS, and more. My experience also extends into Python, making me well suited for your project requiring data scraping and PHP front-end development. Given your project specifics, merging Python for backend tasks like data scraping (using tools like BeautifulSoup or Selenium) and PHP with Laravel for the UI makes perfect sense. I've successfully delivered several projects where real-time data scraping and dynamic UI interaction were crucial, ensuring users have accurate and latest information. For your project, I'd recommend using Python with BeautifulSoup for efficient data scraping. It’s excellent for periodic data retrieval, which fits perfectly for checking and updating the database at set intervals. Laravel, combined with MySQL (MariaDB), will seamlessly handle the front end and database management, providing robust solutions from user authentication to complex SQL query generation via drag and drop interfaces. I would also suggest using AJAX with jQuery or Vue.js for a fluid table viewing and interaction experience, ensuring minimal page reloads and swift content updates. This approach will provide a responsive and visually coherent representation of table relationships and data fields. Additionally, implementing a login system with Laravel’s built-in authentication makes the application secure and manages user sessions effectively. The front end would be equipped with search and filter capabilities to easily navigate through tables and fields—a necessity for large datasets. I'm excited at the prospect of discussing your project further and exploring how we can turn these ideas into reality. I believe with my background and your project needs, we can create something robust and efficient. Looking forward to potentially working together and bringing this project to fruition. Thanks for considering my application!
Copy to Clipboard Apply