Add NikkanGeadai
#689
Add NikkanGeadai
#689
Conversation
|
|
||
| @attribute | ||
| def topics(self) -> List[str]: | ||
| return generic_topic_parsing(self.precomputed.meta.get("keywords")) |
There was a problem hiding this comment.
I feel like the 'keywords' aren't really useful, since they're the same for all articles I've seen:
['Nikkan Gendai DIGITAL', 'Nikkan Gendai', 'Politics', 'Society', 'Entertainment', 'Entertainment', 'Celebrities', 'Sports', 'Column', 'Life', 'Health', 'Business', 'Gendai', 'Gendai', 'gendai', 'Nikkan Gendai']
Instead, I would use the related keywords list, that exists for some articles, e.g. https://www.nikkan-gendai.com/articles/view/news/366688 (search for 関連キーワード)
| for child in element: | ||
| element.remove(child) | ||
| element.tail = None | ||
| element.text = "Test" |
There was a problem hiding this comment.
| element.text = "Test" |
I think this is leftover :)
| @@ -271,6 +271,55 @@ def get_meta_content(root: lxml.html.HtmlElement) -> Dict[str, str]: | |||
| return metadata | |||
|
|
|||
|
|
|||
| def transform_breaks_to_paragraphs(element: lxml.html.HtmlElement, **attribs: str) -> lxml.html.HtmlElement: | |||
There was a problem hiding this comment.
This is a really good idea, but it isn't (yet) viable for TimesOfIndia. Not necessarily all paragraphs are separated with <br> blocks, but also follow <div> blocks for example, which is currently producing issues.
This PR also adds a new utility function to transform
<br>into<p>elements. Maybe @addie9800 wants to give it a shot forTheNamibianand alsoTimesOfIndia.