Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

melee.gg scraping broken #11

Closed
7 tasks done
Badaro opened this issue May 26, 2024 · 3 comments
Closed
7 tasks done

melee.gg scraping broken #11

Badaro opened this issue May 26, 2024 · 3 comments

Comments

@Badaro
Copy link
Owner

Badaro commented May 26, 2024

melee.gg scraping is broken right now.

Good news

  • The main change is not too hard, just a new request that needs to be called to get the decklist.

Bad news

  • This solution adds another request per player, making scraping for melee even slower
  • Decklists for some older tournaments completely vanished, including PT Thunder Junction which I was using for a lot of integration tests, meaning I'll have to rewrite those once again

Edit: As of today (09/Jun) the data for older tournaments is back online and they also fixed it so the extra request is no longer necessary, so I'll resume working on it.

  • Modify code to use the new GetRoundStandings method
  • Review Melee Client tests
  • Review Melee Analyzer tests
  • Review Melee Scraper tests
  • Disable Team tournament scraping as there's no good way to work with those right now (see Re-enable melee.gg team scraping #12 for details)
  • Re-enable Melee Scraper
  • Scrape Melee data from May/15 until now
@Badaro
Copy link
Owner Author

Badaro commented Jun 5, 2024

FYI, since melee seems to be fixing things on their site (decklists for PT Thunder Junction are back, though rounds info is missing and the decklist page is broken) I'll going to wait until things are more stable before making fixes to the scraper.

@Badaro
Copy link
Owner Author

Badaro commented Jun 9, 2024

Rounds data is back online for older tournaments, so I can finally start working on this properly. Adding a checklist with progress.

@Badaro
Copy link
Owner Author

Badaro commented Jun 9, 2024

@Badaro Badaro pinned this issue Jun 12, 2024
@Badaro Badaro closed this as completed Jun 12, 2024
@Badaro Badaro unpinned this issue Jun 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant