Rayrun
BlogAsk AIToolsVideosQA WikiDiscord
← Back to Discord Forum

Different behaviours when trying to automate this website

SeleniumWeb DevelopmentDOM ManipulationHeadless TestingSecurity Testing
x3raphposted in #help-playwright
Open in Discord
x3raph
x3raph

The website in question: https://www.maybank2u.com.my/home/m2u/common/login.do

I have been using Chrome to scrap some data from my account, but recently it stopped working - mainly due to the difference in behaviour when the LOGIN button is clicked.

The website is heavily driven by obfuscated javascript (is this even a good practice?). If you try to log in manually, i.e. after you fill in the username and click LOGIN, it will load a little and show you a security picture verification.

However, when I try to automate it, the website is stuck with the loading animation and the security picture never shows.

Here are some other setups that I have tried:

  • Chrome with "Desktop Chrome" device - not working
  • Chrome with a x, y offset while clicking LOGIN - not working
  • Firefox - works occasionally
  • Firefox with "Desktop Firefox" device - not working
  • Firefox with a x, y offset while clicking LOGIN - works

Here are my questions:

  1. What caused the variability in the behaviour?
  2. What is the best practice to avoid the variability in behaviour? Is there something I could've done better in pw?

Side question: 3. Is this some kind of anti-automation mechanism from Chrome? Or the website itself?

Link to similar case: https://discord.com/channels/807756831384403968/1158628253192425593

security_picture.png
loading.png

This thread is trying to answer question "What is causing the variability in behaviour when trying to automate data scraping from a banking website, and how can it be avoided? Is it an anti-automation mechanism from the website or the browser?"

9 replies
mookiept

offset as in a sleep/wait when/before/after clicking?

x3raph
x3raph
@mookiept: x,y position - otherwise it defaults to the center of the element I think?

x,y position - otherwise it defaults to the center of the element I think?

mookiept

ah, I see. that might help, I guess, given automated tools tend to behave in a very obvious pattern

x3raph
x3raph

They might as well throw in a captcha, I wonder why would they bother coding their own anti-bot mechanism

d3333333

I've worked on few sites that will alternate the login page or other things as a cheap form of captcha, A human would never think twice what is needed and have to troubles... Automation on the other hand will get tripped up... I'd be curious about any end user agreements you clicked on "I agree not to use automation tools to access the site..." Can't think of any site going to much trouble unless you agreed to the terms of service to use their site... What site are you hitting?

refactoreric

Automating websites that don't want to be automated is a cat and mouse game. Probably they don't want to be too friendly to hackers or they want you to pay for using their API (if they have one).

x3raph
x3raph
@d3333333: https://www.maybank2u.com.my/home/m2u/common/login.do I don't think there's anything about automation on their terms and conditions https://www.maybank2u.com.my/WebBank/Terms_and_Conditions_of_Maybank2u_Online_Banking_Services-EN.pdf
tphillips8117

Why are you trying to automate a banking website...? 😛

dand33

@tphillips8117 Exactly, enough said... Those site(s) i automated that changed things up were banks, go figure... 😜

Open in Discord

Related Discord Threads

BlogAsk AIToolsVideosQA WikiDiscord
TwitterGitHubLinkedIn
AboutQuestionsDiscord ForumBrowser ExtensionTagsQA Jobs

Rayrun is a community for QA engineers. I am constantly looking for new ways to add value to people learning Playwright and other browser automation frameworks. If you have feedback, email luc@ray.run.