Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/15: 댓글 수집 이벤트 관련 API 구현-2 #29

Merged
merged 19 commits into from
Aug 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
1660884
Feat: 크롤링 데이터 저장 시, datetime으로 변환되어 저장되도록 변경
yxhwxn Aug 7, 2024
338e8ee
Refactor: 회원가입 시 약관동의 및 유저유형 선택 추가
yxhwxn Aug 7, 2024
0bb7706
Chore: BaseDateTimeEntity 적용
yxhwxn Aug 7, 2024
e09b5ec
Fix: 개발용, 모든 Origin 경로 허용(추후 변경 필요)
yxhwxn Aug 7, 2024
3ec5f8e
Fix: Comment 엔티티 컬럼 수정
yxhwxn Aug 7, 2024
9a65eb7
Feat: 크롤러 관련 예외처리 추가
yxhwxn Aug 7, 2024
85b6984
Fix: allowedOrigins -> allowedOriginPatterens
yxhwxn Aug 7, 2024
a86b400
Fix: 회원가입 API 수정 및 명세 수정
yxhwxn Aug 7, 2024
32da7fb
Refactor: ChromeOption 추가 및 스크롤 시간 증가
yxhwxn Aug 7, 2024
6c1d2d1
Feat: 크롤링된 댓글 리스트 조회 API 구현
yxhwxn Aug 7, 2024
6cc3594
Feat: 사전 URL 검증 API 구현 및 에러케이스 추가
yxhwxn Aug 7, 2024
6c7877e
Chore: ErrorCode 패키지 경로 수정
yxhwxn Aug 7, 2024
d611c78
Feat: 크롤링 데이터 조회 시, 전체 댓글 및 응답 수 추가
yxhwxn Aug 7, 2024
b0a67ad
Fix: 페이지네이션 시작 페이지 번호 수정
yxhwxn Aug 7, 2024
8d5ef9a
Docs: API 명세 수정
yxhwxn Aug 7, 2024
daf2b22
Feat: 조건별 당첨자 랜덤 추첨 리스트 조회 API 구현
yxhwxn Aug 7, 2024
cb3707b
Feat: 키워드별 당첨자 리스트 조회 API 구현
yxhwxn Aug 7, 2024
cadb513
Feat: 당첨자 추첨 리스트 조회 시, 응답 dto 필드 추가
yxhwxn Aug 7, 2024
c1f14fe
Chore: merge dev to feature/15
yxhwxn Aug 7, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,19 +1,70 @@
package com.cmc.suppin.event.crawl.controller;

import com.cmc.suppin.event.crawl.controller.dto.CommentResponseDTO;
import com.cmc.suppin.event.crawl.service.CommentService;
import com.cmc.suppin.global.response.ApiResponse;
import com.cmc.suppin.global.security.reslover.Account;
import com.cmc.suppin.global.security.reslover.CurrentAccount;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.http.ResponseEntity;
import org.springframework.validation.annotation.Validated;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

import java.util.List;

@RestController
@Slf4j
@RequiredArgsConstructor
@Validated
@Tag(name = "Event-Comments", description = "Crawling Comments 관련 API")
@RequestMapping("/api/v1/comments")
@RequestMapping("/api/v1/event/comments")
public class CommentApi {

private final CommentService commentService;

@GetMapping("/list")
@Operation(summary = "크롤링된 전체 댓글 조회 API",
description = "주어진 이벤트 ID와 URL의 댓글을 페이지네이션하여 이벤트의 endDate 전에 작성된 댓글들만 조회합니다.<br><br>" +
"Request: eventId: 조회할 이벤트의 ID, url: 댓글을 조회할 유튜브 URL, page: 조회할 페이지 번호 (1부터 시작), " +
"size: 한 페이지당 댓글 수, Authorization: JWT 토큰을 포함한 인증 헤더<br>" +
"Response: totalCommentCount: 전체 댓글 수, participantCount: 현재 페이지에서 가져온 댓글 수, crawlTime: 댓글 조회(크롤링) 요청 시간, comments: 각 댓글의 상세 정보 배열" +
"author: 댓글 작성자, commentText: 댓글 내용, commentDate: 댓글 작성 시간")
public ResponseEntity<ApiResponse<CommentResponseDTO.CrawledCommentListDTO>> getComments(
@RequestParam Long eventId,
@RequestParam String url,
@RequestParam int page,
@RequestParam int size,
@CurrentAccount Account account) {
CommentResponseDTO.CrawledCommentListDTO comments = commentService.getComments(eventId, url, page, size, account.userId());
return ResponseEntity.ok(ApiResponse.of(comments));
}

@GetMapping("/draft-winners")
@Operation(summary = "조건별 당첨자 추첨 API", description = "주어진 조건에 따라 이벤트의 당첨자를 추첨합니다.")
public ResponseEntity<ApiResponse<CommentResponseDTO.WinnerResponseDTO>> drawWinners(
@RequestParam Long eventId,
@RequestParam String startDate,
@RequestParam String endDate,
@RequestParam int winnerCount,
@RequestParam List<String> keywords,
@CurrentAccount Account account) {
CommentResponseDTO.WinnerResponseDTO winners = commentService.drawWinners(eventId, startDate, endDate, winnerCount, keywords, account.userId());
return ResponseEntity.ok(ApiResponse.of(winners));
}

@GetMapping("/winners/keywordFiltering")
@Operation(summary = "키워드별 당첨자 조회 API", description = "주어진 키워드에 따라 1차 랜덤 추첨된 당첨자 중에서 키워드가 포함된 당첨자들을 조회합니다.")
public ResponseEntity<ApiResponse<List<CommentResponseDTO.CommentDetailDTO>>> getWinnersByKeyword(
@RequestParam Long eventId,
@RequestParam String keyword,
@CurrentAccount Account account) {
List<CommentResponseDTO.CommentDetailDTO> filteredWinners = commentService.getCommentsByKeyword(eventId, keyword, account.userId());
return ResponseEntity.ok(ApiResponse.of(filteredWinners));
}
}
48 changes: 38 additions & 10 deletions src/main/java/com/cmc/suppin/event/crawl/controller/CrawlApi.java
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,7 @@
import lombok.extern.slf4j.Slf4j;
import org.springframework.http.ResponseEntity;
import org.springframework.validation.annotation.Validated;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.bind.annotation.*;

@RestController
@Slf4j
Expand All @@ -26,13 +23,44 @@ public class CrawlApi {

private final CrawlService crawlService;

// 유튜브 크롤링
@GetMapping("/crawling/comments")
@Operation(summary = "유튜브 댓글 크롤링 API", description = "주어진 URL의 유튜브 댓글을 크롤링하고 DB에 저장합니다.")
public ResponseEntity<ApiResponse<Void>> crawlYoutubeComments(@RequestParam String url, @RequestParam Long eventId, @CurrentAccount Account account) {
crawlService.crawlYoutubeComments(url, eventId, account.userId());
return ResponseEntity.ok(ApiResponse.of(ResponseCode.SUCCESS));
// 크롤링 URL 중복 검증
@GetMapping("/comments/checkUrl")
@Operation(summary = "크롤링 중복 검증 API",
description = "주어진 URL과 eventId로 중복된 댓글 수집 이력이 있는지 확인합니다.<br><br>" +
"Request: url: 중복 검증할 URL, eventId: 중복 검증할 이벤트 ID, Authorization: JWT 토큰을 포함한 인증 헤더<br>" +
"Response: 중복된 댓글 수집 이력이 있을 경우 message 출력, 없을 경우 null")
public ResponseEntity<ApiResponse<String>> checkExistingComments(@RequestParam String url, @RequestParam Long eventId, @CurrentAccount Account account) {
String message = crawlService.checkExistingComments(url, eventId, account.userId());
if (message != null) {
return ResponseEntity.ok(ApiResponse.of(ResponseCode.SUCCESS, message));
}
return ResponseEntity.ok(ApiResponse.of(ResponseCode.SUCCESS, "수집 이력이 없습니다."));
}

// 유튜브 댓글 크롤링(DB 저장)
@PostMapping("/crawling/comments")
@Operation(summary = "유튜브 댓글 크롤링 API",
description = "주어진 URL의 유튜브 댓글을 크롤링하여 DB에 저장합니다.<br><br>" +
"Request: url: 크롤링할 URL, eventId: 댓글을 수집할 eventId, forceUpdate: 댓글을 강제로 업데이트할지 여부(Boolean), Authorization: JWT 토큰을 포함한 인증 헤더 <br><br>" +
"forceUpdate 입력 값이 false일 때 설명 <br> " +
"- DB에 기존 댓글이 존재하는 경우: 크롤링을 중지하고 예외를 던집니다. <br>" +
"- DB에 기존 댓글이 존재하지 않는 경우: 새로운 댓글을 크롤링하고 이를 DB에 저장합니다.")
public ResponseEntity<ApiResponse<String>> crawlYoutubeComments(@RequestParam String url, @RequestParam Long eventId, @RequestParam boolean forceUpdate, @CurrentAccount Account account) {
crawlService.crawlYoutubeComments(url, eventId, account.userId(), forceUpdate);
return ResponseEntity.ok(ApiResponse.of(ResponseCode.SUCCESS, "댓글 수집이 완료되었습니다."));
}

// @GetMapping("/count")
// @Operation(summary = "크롤링된 전체 댓글 수 조회 API", description = "주어진 이벤트 ID와 URL의 댓글 수를 조회합니다.<br><br>" +
// "Request: eventId: 조회할 이벤트의 ID, url: 댓글을 조회할 URL, Authorization: JWT 토큰을 포함한 인증 헤더<br>" +
// "Response: 댓글 수")
// public ResponseEntity<ApiResponse<CommentResponseDTO.CommentCountsDTO>> getCommentsCount(
// @RequestParam Long eventId,
// @RequestParam String url,
// @CurrentAccount Account account) {
// int count = commentService.getCommentsCount(eventId, url, account.userId());
// return ResponseEntity.ok(ApiResponse.of(CommentResponseDTO.CommentCountsDTO(count)));
// }

// TODO: 인스타그램 게시글 크롤링
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
package com.cmc.suppin.event.crawl.controller.dto;

import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Getter;
import lombok.NoArgsConstructor;

import java.util.List;

public class CommentRequestDTO {

@Getter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class CommentListRequestDTO {
private Long eventId;
private String url;
private int page;
private int size;
}

@Getter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class WinnerRequestDTO {
private Long eventId;
private String startDate;
private String endDate;
private int winnerCount;
private List<String> keywords;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
package com.cmc.suppin.event.crawl.controller.dto;

import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Getter;
import lombok.NoArgsConstructor;

import java.util.List;

public class CommentResponseDTO {

@Builder
@Getter
@NoArgsConstructor
@AllArgsConstructor
public static class CrawledCommentListDTO {
private int totalCommentCount;
private int participantCount;
private String crawlTime;
private List<CommentDetailDTO> comments;
}

@Getter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class CommentDetailDTO {
private String author;
private String commentText;
private String commentDate;
}

@Getter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class WinnerResponseDTO {
private int winnerCount;
private String startDate;
private String endDate;
private List<CommentDetailDTO> winners;
}
}
Original file line number Diff line number Diff line change
@@ -1,9 +1,13 @@
package com.cmc.suppin.event.crawl.converter;

import com.cmc.suppin.event.crawl.controller.dto.CommentResponseDTO;
import com.cmc.suppin.event.crawl.domain.Comment;
import com.cmc.suppin.event.events.domain.Event;

import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.util.List;
import java.util.stream.Collectors;

public class CommentConverter {

Expand All @@ -16,5 +20,39 @@ public static Comment toCommentEntity(String author, String text, LocalDateTime
.event(event)
.build();
}

public static CommentResponseDTO.CommentDetailDTO toCommentDetailDTO(Comment comment) {
return CommentResponseDTO.CommentDetailDTO.builder()
.author(comment.getAuthor())
.commentText(comment.getCommentText())
.commentDate(comment.getCommentDate().format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm")))
.build();
}

public static CommentResponseDTO.CrawledCommentListDTO toCommentListDTO(List<Comment> comments, String crawlTime, int totalComments) {
List<CommentResponseDTO.CommentDetailDTO> commentDetailDTOS = comments.stream()
.map(CommentConverter::toCommentDetailDTO)
.collect(Collectors.toList());

return CommentResponseDTO.CrawledCommentListDTO.builder()
.totalCommentCount(totalComments)
.participantCount(commentDetailDTOS.size())
.crawlTime(crawlTime)
.comments(commentDetailDTOS)
.build();
}

public static CommentResponseDTO.WinnerResponseDTO toWinnerResponseDTO(List<Comment> winners, int winnerCount, String startDate, String endDate) {
List<CommentResponseDTO.CommentDetailDTO> winnerDetails = winners.stream()
.map(CommentConverter::toCommentDetailDTO)
.collect(Collectors.toList());

return CommentResponseDTO.WinnerResponseDTO.builder()
.winnerCount(winnerCount)
.startDate(startDate)
.endDate(endDate)
.winners(winnerDetails)
.build();
}
}

4 changes: 3 additions & 1 deletion src/main/java/com/cmc/suppin/event/crawl/domain/Comment.java
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@
import lombok.*;
import org.hibernate.annotations.DynamicInsert;

import java.time.LocalDateTime;


@Entity
@Getter
Expand Down Expand Up @@ -34,6 +36,6 @@ public class Comment extends BaseDateTimeEntity {
private String commentText;

@Column(nullable = false)
private String commentDate;
private LocalDateTime commentDate;

}
Original file line number Diff line number Diff line change
@@ -1,12 +1,27 @@
package com.cmc.suppin.event.crawl.domain.repository;

import com.cmc.suppin.event.crawl.domain.Comment;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.Pageable;
import org.springframework.data.jpa.repository.JpaRepository;

import java.time.LocalDateTime;
import java.util.List;

public interface CommentRepository extends JpaRepository<Comment, Long> {
List<Comment> findByEventId(Long eventId);

List<Comment> findByVideoUrl(String videoUrl);
List<Comment> findByUrl(String url);

void deleteByUrlAndEventId(String url, Long eventId);

Page<Comment> findByEventIdAndUrlAndCommentDateBefore(Long eventId, String url, LocalDateTime endDate, Pageable pageable);

List<Comment> findByUrlAndEventId(String url, Long eventId);

int countByEventIdAndUrl(Long eventId, String url);

List<Comment> findByEventIdAndCommentDateBetween(Long eventId, LocalDateTime start, LocalDateTime end);

List<Comment> findByEventIdAndCommentTextContaining(Long eventId, String keyword);
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
package com.cmc.suppin.event.crawl.exception;

import com.cmc.suppin.global.exception.BaseErrorCode;
import com.cmc.suppin.global.response.ErrorResponse;
import lombok.Getter;
import org.springframework.http.HttpStatus;

@Getter
public enum CrawlErrorCode implements BaseErrorCode {

DUPLICATE_URL("crawl-404/01", HttpStatus.CONFLICT, "이미 수집된 댓글이 있는 URL입니다."),
CRAWL_FAILED("crawl-500/01", HttpStatus.INTERNAL_SERVER_ERROR, "크롤링에 실패했습니다.");

private final String code;
private final HttpStatus status;
private final String message;

CrawlErrorCode(String code, HttpStatus status, String message) {
this.code = code;
this.status = status;
this.message = message;
}

@Override
public ErrorResponse getErrorResponse() {
return ErrorResponse.of(code, message);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
package com.cmc.suppin.event.crawl.exception;

import com.cmc.suppin.global.exception.BaseErrorCode;
import com.cmc.suppin.global.exception.CustomException;
import lombok.Getter;

@Getter
public class CrawlException extends CustomException {

public CrawlException(BaseErrorCode errorCode) {
super(errorCode);
}
}
Loading
Loading