Many businesses today heavily rely on web scraping as an essential tool for gathering data from various websites. This technique enables them to systematically extract large volumes of information, which can then be analyzed to gain insights into market trends and competitor strategies. For instance, companies can use web scraping to monitor and compare product prices across different e-commerce platforms, allowing them to adjust their pricing strategies to remain competitive. Additionally, web scraping can help businesses track customer reviews and feedback, providing valuable information to improve their products and services. By leveraging web scraping, businesses can make more informed decisions and stay ahead in the rapidly changing digital landscape.

Technically speaking, up until now, most of us use IPv4. The reason can be summed up in one word: compatibility. However, you might be surprised that things are about to change, it’s our subject for today.

What are IPv4 and IPv6? How they differs?

IPv4 stands for "Internet Protocol." It is a unique identifier used to communicate with and identify machines on the internet. Today, we discuss both IPv4 and IPv6, but originally, only IPv4 existed. It was created in 1981 by DARPA. IPv4 was developed when the internet was in its early stages, so the address space is 32-bit, allowing for a maximum of approximately 4 billion IP addresses (2³²).

If you've been following along, you already know the problem. The internet became so popular that 4 billion IP addresses are not enough. First, because there are now 8 billion people on Earth, and second, because most of us own more than one device. That's precisely why NAT (Network Address Translation) was developed as a workaround to map public and private networks, helping to conserve available IPs. It's very likely that your ISP uses it.

Fast forward to 1995, the second version of the IP standard, IPv6, was developed. This version uses a 128-bit address space, allowing for 340 undecillion IP addresses, or in other words, 340 billion billion billion billion, four times over.

It's now clear that this should be enough for any population on Earth and any number of devices each of us might own. However, there's one problem: some websites are not compatible with IPv6 yet.

The pros of IPv6

Since 1995, many website adopted IPv6, according to IPv6 Adoption Statistics by Google, 43% of websites are supporting IPv6 today.

From a web scraping perspective, IPv6 offers many advantages:

  • Larger IP pool: With the IPv6 address space, the number of IPs is so vast that individuals and companies like ours can receive billions of IPs. Generally, the authority that assigns subnets, ARIN, provides these details:
    • Individuals can get a /64 subnet, which includes 18 quintillion IPs.
    • Businesses, depending on their size, can receive a subnet between /48 and /32.
  • Lower cost: Because the address space is so large, there isn't the scarcity that IPv4 has today. Some cloud providers now offer servers with multiple IPv6 addresses at a fraction of the cost.
  • Improved anonymity: Today, some companies specialize in fraud and bot detection, often using databases to track suspicious IP addresses. However, applying the same method to IPv6 can be challenging and might require blocking entire subnets, which could be ineffective. Additionally, since IPv6 is not widely used yet, most of its IP addresses have never been used before. Unlike IPv4, there is no need for IP recycling.

Cons of IPv6

The only downside of IPv6 is its lack of compatibility; currently, only 43% of websites and networks support IPv6, which might seem too low for some people. However, this is changing quickly, especially with major players like Cloudflare, which has been enforcing IPv6 on all websites for a few months now. Only Enterprise accounts can disable it; previously, it was an opt-in option.

Our recommendations

Web scraping using IPv6 might not be suitable for everyone, but it's worth considering, especially with the rising cost of quality IPv4 proxies.

If you're thinking about buying proxies, first check if your target website supports IPv6. You can do this easily with the IPv6 Compatibility Test on our website or by using the command line:

  • On macOS and Linux: dig example.com AAAA
  • On Windows: nslookup -type=AAAA example.com

We recommend using our IPv6 Compatibility Test, as your local network might not support IPv6. You can perform the test here: test-ipv6.com. Don't worry, AltV6 acts as an IPv4 to IPv6 tunnel, so you can still run scraping from your network, and we'll redirect it to IPv6.

If your website shows AAAA records, then it supports IPv6.

Once you have identified a website that supports IPv6, we recommend trying our introductory offer at $5 per month. This plan provides 5 gigabytes of traffic, which should be more than sufficient to get started!