playwright not looping
page.click('#dnn_ctr18001_Login_Login_EuclidCVCustom_btnLogin') page.wait_for_timeout(5000) page.goto('https://www.connexfm.com/Directories/Multi-Site-FM-Directory') page.wait_for_load_state('load') frame = page.frame(name='dnn_ctr18371_IFrame_htmIFrame') frame.wait_for_load_state('load') search_button_selector = 'button.btn.btn-primary' frame.click(search_button_selector) frame.wait_for_load_state('networkidle') link_elements = frame.query_selector_all('a')
print(link_elements)
for link_element in link_elements:
if link_element.is_visible():
link_element.click()
# Wait for some time to allow the new page to load completely
frame.wait_for_load_state('load')
page.wait_for_timeout(5000)after scraping first link element is not looping to the 2nd
This thread is trying to answer question "Why is Playwright not looping through all link elements after scraping the first one?"
Hi, first of all, don't use any methods which are working with element handles or frames directly. Those are a leftover from the Puppeteer heritage of Playwright. Nowadays there are more reliable ways to work with elements and frames:
The Locator and FrameLocator. https://playwright.dev/python/docs/api/class-locator https://playwright.dev/python/docs/api/class-framelocator
Then to go to the core of the problem: Possibly/probably the load event is emitted but the app loads more details using asynchronous API calls.
Try to find some indication of page readiness, and wait for it explicitly before starting your loop. https://playwright.dev/python/docs/api/class-locator#locator-wait-for https://playwright.dev/python/docs/test-assertions#list-of-assertions https://playwright.dev/python/docs/events#waiting-for-event
Also you are navigating to a new page, probably in the same tab as where your list of links is present. When you navigate to the new page, your link element handles are probably going stale.
Probably you should open (and close) the various destination pages in a separate tab.
Either by using a CTRL modifier to open the link in a new tab on clicking, and then intercepting the 'page' event: https://playwright.dev/python/docs/api/class-locator#locator-click-option-modifiers https://playwright.dev/python/docs/api/class-browsercontext#browser-context-event-page
Or by reading the link href, opening a new page, and then navigating that new page to the link href: https://playwright.dev/python/docs/api/class-browser#browser-new-page
Rayrun is a community for QA engineers. I am constantly looking for new ways to add value to people learning Playwright and other browser automation frameworks. If you have feedback, email luc@ray.run.