Hello Selenium

Introduction

I would like to crawl some data from Mods For Melon Playground, but I have no experience with it. Consider using Selenium, a powerful tool for web development and data extraction. In this series, I’ll delve into the basics of Selenium to automate data extraction from the site.

Hello Selenium

Setting Up the Project

To embark on our Selenium journey, we’ll start by setting up a new project using IntelliJ. Follow these steps:

Create a new project in IntelliJ.

Add Dependencies. Open the build.gradle.kts file and include the following dependencies:

1
2


testImplementation("org.seleniumhq.selenium:selenium-java:4.15.0")
testImplementation("org.junit.jupiter:junit-jupiter-engine:5.10.0")

Update Kotlin Version. Change kotlin in the build.gradle.kts file to ensure compatibility:
1 2 3

kotlin { jvmToolchain(18) }

Writing Your First Selenium Script

Now, let’s dive into writing a simple Selenium script. Create a new file at src/test/kotlin/dev/selenium with the following content:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44


package dev.selenium

import org.junit.jupiter.api.*
import org.junit.jupiter.api.Assertions.assertEquals
import org.openqa.selenium.By
import org.openqa.selenium.WebDriver
import org.openqa.selenium.chrome.ChromeDriver
import java.time.Duration

@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class FirstScriptTest {
    private lateinit var driver: WebDriver

    @Test
    fun eightComponents() {
        driver = ChromeDriver()

        // Navigate to the target website
        driver.get("https://www.selenium.dev/selenium/web/web-form.html")

        // Verify the title of the page
        val title = driver.title
        assertEquals("Web form", title)

        // Set an implicit wait to handle dynamic content loading
        driver.manage().timeouts().implicitlyWait(Duration.ofMillis(500))

        // Locate the text box and submit button
        var textBox = driver.findElement(By.name("my-text"))
        val submitButton = driver.findElement(By.cssSelector("button"))

        // Input text and submit the form
        textBox.sendKeys("Selenium")
        submitButton.click()

        // Verify the result
        val message = driver.findElement(By.id("message"))
        val value = message.text
        assertEquals("Received!", value)

        // Quit the browser
        driver.quit()
    }
}

Running the Script

With your script in place, it’s time to witness the magic. Execute the script, and watch as the Chrome browser automagically performs the specified steps on the target website.

Congratulations! You’ve just taken your first steps into the world of Selenium. This is just the beginning – as you explore further, you’ll discover the immense capabilities Selenium offers for web scraping and automation. Stay tuned for more adventures in web development and data extraction!