SpringBoot集成ElasticSearch實(shí)現(xiàn)minio文件內(nèi)容全文檢索
一、docker安裝Elasticsearch
(1)springboot和Elasticsearch的版本對應(yīng)關(guān)系如下,請看版本對應(yīng):
注意安裝對應(yīng)版本,否則可能會(huì)出現(xiàn)一些未知的錯(cuò)誤。
(2)拉取鏡像
docker pull elasticsearch:7.17.6
(3)運(yùn)行容器
docker run -it -d --name elasticsearch -e "discovery.type=single-node" -e "ES_JAVA_OPTS=-Xms512m -Xmx1024m" -p 9200:9200 -p 9300:9300 elasticsearch:7.17.6
訪問http://localhost:9200/,出現(xiàn)如下內(nèi)容表示安裝成功。
(4)安裝中文分詞器
進(jìn)入容器:
docker exec -it elasticsearch bash
然后進(jìn)入bin目錄執(zhí)行下載安裝ik分詞器命令:
elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.17.6/elasticsearch-analysis-ik-7.17.6.zip
退出bash并重啟容器:
docker restart elasticsearch
二、安裝kibana
Kibana 是為 Elasticsearch設(shè)計(jì)的開源分析和可視化平臺。你可以使用 Kibana 來搜索,查看存儲在 Elasticsearch 索引中的數(shù)據(jù)并與之交互。你可以很容易實(shí)現(xiàn)高級的數(shù)據(jù)分析和可視化,以圖表的形式展現(xiàn)出來。
(1)拉取鏡像
docker pull kibana:7.17.6
(2)運(yùn)行容器
docker run --name kibana -p 5601:5601 --link elasticsearch:es -e "elasticsearch.hosts=http://es:9200" -d kibana:7.17.6
--link elasticsearch:es表示容器互聯(lián),即容器kibana連接到elasticsearch。
(3)使用kibana dev_tools發(fā)送http請求操作Elasticsearch
三、后端代碼
(1)引入maven依賴
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-elasticsearch</artifactId> </dependency>
(2)application.yml配置
spring: elasticsearch: uris: http://localhost:9200
(3)實(shí)體類
import lombok.AllArgsConstructor; import lombok.Data; import lombok.NoArgsConstructor; import org.springframework.data.annotation.Id; import org.springframework.data.elasticsearch.annotations.Document; import org.springframework.data.elasticsearch.annotations.Field; import org.springframework.data.elasticsearch.annotations.FieldType; import java.util.Date; /** * @author yangfeng */ @Data @NoArgsConstructor @AllArgsConstructor @Document(indexName = "file") public class File { @Id private String id; /** * 文件名稱 */ @Field(type = FieldType.Text, analyzer = "ik_max_word") private String fileName; /** * 文件分類 */ @Field(type = FieldType.Keyword) private String fileCategory; /** * 文件內(nèi)容 */ @Field(type = FieldType.Text, analyzer = "ik_max_word") private String fileContent; /** * 文件存儲路徑 */ @Field(type = FieldType.Keyword, index = false) private String filePath; /** * 文件大小 */ @Field(type = FieldType.Keyword, index = false) private Long fileSize; /** * 文件類型 */ @Field(type = FieldType.Keyword, index = false) private String fileType; /** * 創(chuàng)建人 */ @Field(type = FieldType.Keyword, index = false) private String createBy; /** * 創(chuàng)建日期 */ @Field(type = FieldType.Keyword, index = false) private Date createTime; /** * 更新人 */ @Field(type = FieldType.Keyword, index = false) private String updateBy; /** * 更新日期 */ @Field(type = FieldType.Keyword, index = false) private Date updateTime; }
(4)repository接口,繼承ElasticsearchRepository
import org.springframework.data.domain.Page; import org.springframework.data.domain.Pageable; import org.springframework.data.elasticsearch.annotations.Highlight; import org.springframework.data.elasticsearch.annotations.HighlightField; import org.springframework.data.elasticsearch.annotations.HighlightParameters; import org.springframework.data.elasticsearch.core.SearchHit; import org.springframework.data.elasticsearch.repository.ElasticsearchRepository; import org.springframework.stereotype.Repository; import java.util.List; /** * @author yangfeng * @date: 2024年11月9日 15:29 */ @Repository public interface FileRepository extends ElasticsearchRepository<File, String> { /** * 關(guān)鍵字查詢 * * @return */ @Highlight(fields = {@HighlightField(name = "fileName"), @HighlightField(name = "fileContent")}, parameters = @HighlightParameters(preTags = {"<span style='color:red'>"}, postTags = {"</span>"}, numberOfFragments = 0)) List<SearchHit<File>> findByFileNameOrFileContent(String fileName, String fileContent, Pageable pageable); }
(5)service接口
import org.springframework.data.elasticsearch.core.SearchHit; import org.springframework.data.elasticsearch.core.SearchHits; import java.util.List; /** * description: ES文件服務(wù) * * @author yangfeng * @version V1.0 * @date 2023-02-21 */ public interface IFileService { /** * 保存文件 */ void saveFile(String filePath, String fileCategory) throws Exception; /** * 關(guān)鍵字查詢 * * @return */ List<SearchHit<File>> search(FileDTO dto); /** * 關(guān)鍵字查詢 * * @return */ SearchHits<File> searchPage(FileDTO dto); }
(6)service實(shí)現(xiàn)類
import cn.hutool.core.util.IdUtil; import lombok.extern.slf4j.Slf4j; import org.apache.commons.lang3.StringUtils; import org.apache.shiro.SecurityUtils; import org.elasticsearch.index.query.QueryBuilders; import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder; import org.elasticsearch.search.sort.SortBuilders; import org.elasticsearch.search.sort.SortOrder; import org.jeecg.common.exception.JeecgBootException; import org.jeecg.common.system.vo.LoginUser; import org.jeecg.common.util.CommonUtils; import org.jeecg.common.util.MinioUtil; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.data.domain.PageRequest; import org.springframework.data.domain.Pageable; import org.springframework.data.domain.Sort; import org.springframework.data.elasticsearch.core.ElasticsearchRestTemplate; import org.springframework.data.elasticsearch.core.SearchHit; import org.springframework.data.elasticsearch.core.SearchHits; import org.springframework.data.elasticsearch.core.query.NativeSearchQuery; import org.springframework.data.elasticsearch.core.query.NativeSearchQueryBuilder; import org.springframework.stereotype.Service; import java.io.InputStream; import java.util.Date; import java.util.List; import java.util.Objects; /** * description: ES文件服務(wù) * * @author yangfeng * @version V1.0 * @date 2023-02-21 */ @Slf4j @Service public class FileServiceImpl implements IFileService { @Autowired private FileRepository fileRepository; @Autowired private ElasticsearchRestTemplate elasticsearchRestTemplate; /** * 保存文件 */ @Override public void saveFile(String filePath, String fileCategory) throws Exception { if (Objects.isNull(filePath)) { throw new JeecgBootException("文件不存在"); } LoginUser user = (LoginUser) SecurityUtils.getSubject().getPrincipal(); String fileName = CommonUtils.getFileNameByUrl(filePath); String fileType = StringUtils.isNotBlank(fileName) ? fileName.substring(fileName.lastIndexOf(".") + 1) : null; InputStream inputStream = MinioUtil.getMinioFile(filePath); // 讀取文件內(nèi)容,上傳到es,方便后續(xù)的檢索 String fileContent = FileUtils.readFileContent(inputStream, fileType); File file = new File(); file.setId(IdUtil.getSnowflake(1, 1).nextIdStr()); file.setFileContent(fileContent); file.setFileName(fileName); file.setFilePath(filePath); file.setFileType(fileType); file.setFileCategory(fileCategory); file.setCreateBy(user.getUsername()); file.setCreateTime(new Date()); fileRepository.save(file); } /** * 關(guān)鍵字查詢 * * @return */ @Override public List<SearchHit<File>> search(FileDTO dto) { Pageable pageable = PageRequest.of(dto.getPageNo() - 1, dto.getPageSize(), Sort.Direction.DESC, "createTime"); return fileRepository.findByFileNameOrFileContent(dto.getKeyword(), dto.getKeyword(), pageable); } @Override public SearchHits<File> searchPage(FileDTO dto) { NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder(); queryBuilder.withQuery(QueryBuilders.multiMatchQuery(dto.getKeyword(), "fileName", "fileContent")); // 設(shè)置高亮 HighlightBuilder highlightBuilder = new HighlightBuilder(); String[] fieldNames = {"fileName", "fileContent"}; for (String fieldName : fieldNames) { highlightBuilder.field(fieldName); } highlightBuilder.preTags("<span style='color:red'>"); highlightBuilder.postTags("</span>"); highlightBuilder.order(); queryBuilder.withHighlightBuilder(highlightBuilder); // 也可以添加分頁和排序 queryBuilder.withSorts(SortBuilders.fieldSort("createTime").order(SortOrder.DESC)) .withPageable(PageRequest.of(dto.getPageNo() - 1, dto.getPageSize())); NativeSearchQuery nativeSearchQuery = queryBuilder.build(); return elasticsearchRestTemplate.search(nativeSearchQuery, File.class); } }
(7)controller
import lombok.extern.slf4j.Slf4j; import org.jeecg.common.api.vo.Result; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.web.bind.annotation.PostMapping; import org.springframework.web.bind.annotation.RequestBody; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RestController; /** * 文件es操作 * * @author yangfeng * @since 2024-11-09 */ @Slf4j @RestController @RequestMapping("/elasticsearch/file") public class FileController { @Autowired private IFileService fileService; /** * 保存文件 * * @return */ @PostMapping(value = "/saveFile") public Result<?> saveFile(@RequestBody File file) throws Exception { fileService.saveFile(file.getFilePath(), file.getFileCategory()); return Result.OK(); } /** * 關(guān)鍵字查詢-repository * * @throws Exception */ @PostMapping(value = "/search") public Result<?> search(@RequestBody FileDTO dto) { return Result.OK(fileService.search(dto)); } /** * 關(guān)鍵字查詢-原生方法 * * @throws Exception */ @PostMapping(value = "/searchPage") public Result<?> searchPage(@RequestBody FileDTO dto) { return Result.OK(fileService.searchPage(dto)); } }
(8)工具類
import lombok.extern.slf4j.Slf4j; import org.apache.pdfbox.pdmodel.PDDocument; import org.apache.pdfbox.text.PDFTextStripper; import org.apache.poi.xwpf.extractor.XWPFWordExtractor; import org.apache.poi.xwpf.usermodel.XWPFDocument; import java.io.IOException; import java.io.InputStream; import java.io.InputStreamReader; import java.nio.charset.StandardCharsets; import java.util.Arrays; import java.util.List; @Slf4j public class FileUtils { private static final List<String> FILE_TYPE; static { FILE_TYPE = Arrays.asList("pdf", "doc", "docx", "text"); } public static String readFileContent(InputStream inputStream, String fileType) throws Exception{ if (!FILE_TYPE.contains(fileType)) { return null; } // 使用PdfBox讀取pdf文件內(nèi)容 if ("pdf".equalsIgnoreCase(fileType)) { return readPdfContent(inputStream); } else if ("doc".equalsIgnoreCase(fileType) || "docx".equalsIgnoreCase(fileType)) { return readDocOrDocxContent(inputStream); } else if ("text".equalsIgnoreCase(fileType)) { return readTextContent(inputStream); } return null; } private static String readPdfContent(InputStream inputStream) throws Exception { // 加載PDF文檔 PDDocument pdDocument = PDDocument.load(inputStream); // 創(chuàng)建PDFTextStripper對象, 提取文本 PDFTextStripper textStripper = new PDFTextStripper(); // 提取文本 String content = textStripper.getText(pdDocument); // 關(guān)閉PDF文檔 pdDocument.close(); return content; } private static String readDocOrDocxContent(InputStream inputStream) { try { // 加載DOC文檔 XWPFDocument document = new XWPFDocument(inputStream); // 2. 提取文本內(nèi)容 XWPFWordExtractor extractor = new XWPFWordExtractor(document); return extractor.getText(); } catch (IOException e) { e.printStackTrace(); return null; } } private static String readTextContent(InputStream inputStream) { StringBuilder content = new StringBuilder(); try (InputStreamReader isr = new InputStreamReader(inputStream, StandardCharsets.UTF_8)) { int ch; while ((ch = isr.read()) != -1) { content.append((char) ch); } } catch (IOException e) { e.printStackTrace(); return null; } return content.toString(); } }
(9)dto
import lombok.Data; @Data public class FileDTO { private String keyword; private Integer pageNo; private Integer pageSize; }
四、前端代碼
(1)查詢組件封裝
<template> <a-input-search v-model:value="pageInfo.keyword" placeholder="全文檢索" @search="handleSearch" style="width: 220px;margin-left:30px" /> <a-modal v-model:visible="showSearch" title="全文檢索" width="900px" :footer="null" destroy-on-close> <SearchContent :items="searchItems" :loading="loading"/> <div style="padding: 10px;display: flex;justify-content: flex-end"> <Pagination v-if="pageInfo.total" :pageSize="pageInfo.pageSize" :pageNo="pageInfo.pageNo" :total="pageInfo.total" @pageChange="changePage" :show-total="total => `共 ${total} 條`"/> </div> </a-modal> </template> <script lang="ts" setup> import {ref} from 'vue' import {Pagination} from "ant-design-vue"; import SearchContent from "@/components/ElasticSearch/SearchContent.vue" import {searchPage} from "@/api/sys/elasticsearch" const loading = ref<boolean>(false) const showSearch = ref<any>(false) const searchItems = ref<any>(); const pageInfo = ref<{ pageNo: number; pageSize: number; keyword: string; total: number; }>({ // 當(dāng)前頁碼 pageNo: 1, // 當(dāng)前每頁顯示多少條數(shù)據(jù) pageSize: 10, keyword: '', total: 0, }); async function handleSearch() { if (!pageInfo.value.keyword) { return; } pageInfo.value.pageNo = 1 showSearch.value = true await getSearchItems(); } function changePage(pageNo) { pageInfo.value.pageNo = pageNo getSearchItems(); } async function getSearchItems() { loading.value = true try { const res: any = await searchPage(pageInfo.value); searchItems.value = res?.searchHits; debugger pageInfo.value.total = res?.totalHits } finally { loading.value = false } } </script> <style scoped></style>
(2)接口elasticsearch.ts
import {defHttp} from '/@/utils/http/axios'; enum Api { saveFile = '/elasticsearch/file/saveFile', searchPage = '/elasticsearch/file/searchPage', } /** * 保存文件到es * @param params */ export const saveFile = (params) => defHttp.post({ url: Api.saveFile, params }); /** * 關(guān)鍵字查詢-原生方法 * @param params */ export const searchPage = (params) => defHttp.post({ url: Api.searchPage, params },);
(3)搜索內(nèi)容組件SearchContent.vue
<template> <a-spin :spinning="loading"> <div class="searchContent"> <div v-for="(item,index) in items" :key="index" v-if="!!items.length > 0"> <a-card class="contentCard"> <template #title> <a @click="detailSearch(item.content)"> <div class="flex" style="align-items: center"> <div> <img src="../../assets/images/pdf.png" v-if="item?.content?.fileType=='pdf'" style="width: 20px"/> <img src="../../assets/images/word.png" v-if="item?.content?.fileType=='word'" style="width: 20px"/> <img src="../../assets/images/excel.png" v-if="item?.content?.fileType=='excel'" style="width: 20px"/> </div> <div style="margin-left:10px"> <article class="article" v-html="item.highlightFields.fileName" v-if="item?.highlightFields?.fileName"></article> <span v-else>{{ item?.content?.fileName }}</span> </div> </div> </a> </template> <div class="item"> <article class="article" v-html="item.highlightFields.fileContent" v-if="item?.highlightFields?.fileContent"></article> <span v-else>{{ item?.content?.fileContent?.length > 150 ? item.content.fileContent.substring(0, 150) + '......' : item.content.fileContent }}</span> </div> </a-card> </div> <EmptyData v-else/> </div> </a-spin> </template> <script lang="ts" setup> import {useGlobSetting} from "@/hooks/setting"; import EmptyData from "/@/components/ElasticSearch/EmptyData.vue"; import {ref} from "vue"; const glob = useGlobSetting(); const props = defineProps({ loading: { type: Boolean, default: false }, items: { type: Array, default: [] }, }) function detailSearch(searchItem) { const url = ref(`${glob.domainUrl}/sys/common/pdf/preview/`); window.open(url.value + searchItem.filePath + '#scrollbars=0&toolbar=0&statusbar=0', '_blank'); } </script> <style lang="less" scoped> .searchContent { min-height: 500px; overflow-y: auto; } .contentCard { margin: 10px 20px; } a { color: black; } a:hover { color: #3370ff; } :deep(.ant-card-body) { padding: 13px; } </style>
五、效果展示
以上就是SpringBoot集成ElasticSearch實(shí)現(xiàn)minio文件內(nèi)容全文檢索的詳細(xì)內(nèi)容,更多關(guān)于SpringBoot minio內(nèi)容全文檢索的資料請關(guān)注腳本之家其它相關(guān)文章!
相關(guān)文章
java項(xiàng)目導(dǎo)出為.exe執(zhí)行文件的方法步驟
最近做了個(gè)項(xiàng)目,想要轉(zhuǎn)換成可執(zhí)行文件,那么java項(xiàng)目如何導(dǎo)出為.exe執(zhí)行文件,本文就介紹一下,主要使用jar2exe軟件,感興趣的可以了解一下2021-05-05你應(yīng)該知道的21個(gè)Java核心技術(shù)
Java的21個(gè)核心技術(shù)點(diǎn),你知道嗎?這篇文章主要為大家詳細(xì)介紹了Java核心技術(shù),具有一定的參考價(jià)值,感興趣的小伙伴們可以參考一下2017-08-08SpringBoot整合JWT(JSON?Web?Token)生成token與驗(yàn)證的流程及示例
JSON Web Token(JWT)是一種開放的標(biāo)準(zhǔn)(RFC 7519),定義了一種緊湊的、自包含的方式來安全地在各方之間傳輸信息作為JSON對象,這篇文章主要給大家介紹了關(guān)于SpringBoot整合JWT(JSON?Web?Token)生成token與驗(yàn)證的相關(guān)資料,需要的朋友可以參考下2024-07-07Java Runtime類詳解_動(dòng)力節(jié)點(diǎn)Java學(xué)院整理
Runtime類封裝了運(yùn)行時(shí)的環(huán)境。每個(gè) Java 應(yīng)用程序都有一個(gè) Runtime 類實(shí)例,使應(yīng)用程序能夠與其運(yùn)行的環(huán)境相連接。下面通過本文給大家分享Java Runtime類詳解,需要的朋友參考下吧2017-04-04淺析JAVA中的內(nèi)存結(jié)構(gòu)、重載、this與繼承
這篇文章主要介紹了 JAVA中的內(nèi)存結(jié)構(gòu)、重載、this與繼承的相關(guān)資料,本文給大家介紹的非常詳細(xì),對大家的學(xué)習(xí)或工作具有一定的參考借鑒價(jià)值,需要的朋友可以參考下2021-03-03關(guān)于Synchronized和ReentranLock的區(qū)別及說明
文章介紹了Java中的`synchronized`關(guān)鍵字和`ReentrantLock`類,兩者都可以用于解決多線程同步問題,但`ReentrantLock`提供了更多的功能和靈活性2024-12-12親測SpringBoot參數(shù)傳遞及@RequestBody注解---踩過的坑及解決
這篇文章主要介紹了親測SpringBoot參數(shù)傳遞及@RequestBody注解---踩過的坑及解決方案,具有很好的參考價(jià)值,希望對大家有所幫助。如有錯(cuò)誤或未考慮完全的地方,望不吝賜教2021-10-10