Automating Google Search with Python Selenium
Automating Google Search with Python Selenium - My Automation Journey Begins
This project the beginning of my journey into Selenium Automation Testing. Until now, I had only heard about automation as a powerful way to simplify repetitive digital tasks. But today, I finally took the first step—understanding what Selenium is, why it exists, and how it can empower us as developers.
What is Selenium?
Selenium is a web automation tool that allows us to control web browsers programmatically.
Instead of performing every click, search, or form submission manually, Selenium lets us write Python code that does the same actions automatically. It supports different browsers like Chrome, Firefox, and Edge, making browser automation possible across platforms.
This automation is especially useful when dealing with repetitive tasks, and that’s exactly where my learning journey began.
The Situation That Sparked This Idea
As students, we often face the same boring challenge: writing lengthy assignments. We're given a list of questions, and we end up copying each one, searching it online, reading multiple sources, and finally writing the answers. The most repetitive part of this entire process is simple—copying and searching, again and again.
This made me think:
Why not build a small tool that automates just the “searching” part? Something I can use across subjects and even in the future.
When I explored this idea, I came across Selenium. It immediately made sense—Selenium can automate everything I was manually doing inside the browser.
Understanding the Components: File Handling + Selenium WebDriver
The assignment questions come in a document file. So the first problem was:
How do I extract each question from the file?
Python’s file-handling abilities were perfect for this. By opening the file in read (r) mode, I could read the entire content line by line and store each question in a list. Lists support indexing and iteration, so they are ideal for processing multiple questions one after another.
Next comes the main engine of the project—the Selenium WebDriver.
The WebDriver interacts directly with the browser. It sends HTTP commands to the browser’s DevTools protocol, which interprets these commands as real user actions such as loading pages, typing, clicking, or opening new tabs.
With this setup, I could iterate over my list of questions, open the browser, locate the search bar, type each question, and perform the search automatically.
How I Structured the Program
I began by importing the essential Selenium modules such as webdriver, By, Options, and Keys. After that, I wrote a simple logic to:
-
Open and read the document.
-
Extract each question into a Python list.
-
Initialize the Selenium WebDriver with proper configurations.
-
Loop through the questions and:
-
Open a new browser tab
-
Locate the Google search bar
-
Type the question
-
Trigger a search
-
This foundation helped me understand how Selenium controls the browser step-by-step.
The First Major Error: Browser Detection
While running my automation script, I faced a common challenge:
Google detected that Chrome was being controlled by an automated tool. ( I tried on both Edge, Chrome )
Modern websites use bot-detection (like Google's reCAPTCHA), so Selenium-controlled sessions are often flagged and redirected to CAPTCHA verification. This stops the automation flow.
To handle this, I used a simple but effective technique—setting a custom user-agent.
By modifying the WebDriver Options, I made the browser appear more like a regular user instead of an automated bot.
Example:
options.add_argument(
"user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36"
)
This doesn’t completely eliminate detection issues but reduces them significantly for learning and small-scale automation.
Key Takeaways 🎯
Comments
Post a Comment