Skip to content

Commit

Permalink
change file-storage
Browse files Browse the repository at this point in the history
  • Loading branch information
rbmonster committed Oct 8, 2021
1 parent 6bef12a commit 1aebdfb
Show file tree
Hide file tree
Showing 74 changed files with 488 additions and 488 deletions.
4 changes: 2 additions & 2 deletions src/main/java/com/bigdata/SPARK.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,11 +38,11 @@ scala>
```

- master
![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/learning/basic/sparkMaster.jpg)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/learning/basic/sparkMaster.jpg)


- submit job
![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/learning/basic/sparkWoker.jpg)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/learning/basic/sparkWoker.jpg)



Expand Down
6 changes: 3 additions & 3 deletions src/main/java/com/design/RPC_DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,13 +86,13 @@ Server: Apache 0.84
- 服务消费者 Service Consumer,客户端需要有从注册中心获取服务的基本能力,它需要在应用启动时,扫描依赖的RPC服务,并为其生成代理调用对象,同时从注册中心拉取服务元数据存入本地缓存,然后发起监听各服务的变动做到及时更新缓存。

围绕上面三个基本要素可以进一步扩展服务路由、负载均衡、服务熔断降级、序列化协议、通信协议等等。
![image](https://github.com/rbmonster/file-storage/blob/main/learning-note/other/rpc.png)
![image](https://gitee.com/rbmon/file-storage/raw/main/learning-note/other/rpc.png)


### 技术选型
#### 注册中心
目前成熟的注册中心有Zookeeper,Nacos,Consul,Eureka,它们的主要比较如下:
![image](https://github.com/rbmonster/file-storage/blob/main/learning-note/other/rpcRegister.png)
![image](https://gitee.com/rbmon/file-storage/raw/main/learning-note/other/rpcRegister.png)

#### IO通信框架
Netty、MINA等等均可以作为底层通信框架,目前使用Netty作为通信框架的居多
Expand All @@ -111,7 +111,7 @@ TCP通信过程中会根据TCP缓冲区的实际情况进行包的划分,所
目前有多种的序列化协议如JavaSerializer、Protobuf及Hessian。建议选用Protobuf,其序列化后码流小性能高,非常适合RPC调用,Google自家的gRPC也是用其作为通信协议。

### 参考架构
![image](https://github.com/rbmonster/file-storage/blob/main/learning-note/other/rpcDesign.png)
![image](https://gitee.com/rbmon/file-storage/raw/main/learning-note/other/rpcDesign.png)

## Java基于Spring实现思路
注册中心使用zookeeper,通信框架使用Netty
Expand Down
2 changes: 1 addition & 1 deletion src/main/java/com/design/SECONDS_KILL_DESIGN.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# 秒杀系统
![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/design/secondKill.jpg)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/design/secondKill.jpg)

## 前端
秒杀场景入口商城网页、H5、APP、小程序
Expand Down
4 changes: 2 additions & 2 deletions src/main/java/com/design/SENSITIVE_FILTER.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,13 @@ java中只要是字符,不管是数字还是英文还是汉字,都占两个
完全可以使用内存的方案来解决,即直接将所有词汇load到内存中。

假如存在上千万条的词汇,那么在上述数据量 1.9M*1000*n 会需要几个G的内存资源就要考虑非内存的其他的方案。
![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/design/character.jpg)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/design/character.jpg)


## DFA算法 (Deterministic Finite Automaton)
基本思想是基于状态转移来检索敏感词,只需要扫描一次待检测文本,就能对所有敏感词进行检测。

![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/design/sensitiveStructure.jpg)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/design/sensitiveStructure.jpg)
将敏感词库中相同前缀的词构建成了一个树形结构

在java 中以HashMap结构进行存储
Expand Down
6 changes: 3 additions & 3 deletions src/main/java/com/design/SYSTEM-DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,9 +50,9 @@ MVC是一种框架模式。经典MVC模式中,M是指业务模型,V是指用

##### 分层架构

![image](https://github.com/rbmonster/file-storage/blob/main/learning-note/design/systemdesign/structure.png)
![image](https://gitee.com/rbmon/file-storage/raw/main/learning-note/design/systemdesign/structure.png)

![image](https://github.com/rbmonster/file-storage/blob/main/learning-note/design/systemdesign/structureIntroduce.png)
![image](https://gitee.com/rbmon/file-storage/raw/main/learning-note/design/systemdesign/structureIntroduce.png)

- 应用层(接入层):通常用来接收前端(展现层)的请求,转发给领域层获取请求结果,再组装结果返回前端。
- 基础设施层:作为其他层支撑的存在,最通俗的例子就是searchServ,正常的搜索服务都会集成ElasticSearch或者其他搜索功能,searchSearch封装了与基础服务集成的及细节,只暴露了领域需要的接口。
Expand All @@ -78,7 +78,7 @@ MVC是一种框架模式。经典MVC模式中,M是指业务模型,V是指用
> 这个根是一个实体,并且它是外部可以访问的唯一的对象。根可以保持对任意聚合对象的引用,并且其他的对象可以持有任意其他的对象,但一个外部对象只能持有根对象的引用。如果边界内有其他的实体,那些实体的标识符是本地化的,只在聚合内有意义。
一个简单的聚合的案例如下图所示。客户是聚合的根,并且其他所有的对象都是内部的。如果需要地址,一个它的拷贝将被传递到外部对象。
![image](https://github.com/rbmonster/file-storage/blob/main/learning-note/design/systemdesign/aggreate.png)
![image](https://gitee.com/rbmon/file-storage/raw/main/learning-note/design/systemdesign/aggreate.png)


###### 工厂
Expand Down
2 changes: 1 addition & 1 deletion src/main/java/com/four/FILTERANDINTERCEPTOR.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# MVC 拦截器与过滤器

![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/four/filter2intercept.jpg)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/four/filter2intercept.jpg)
## 过滤器 servlet
filter(过滤器)作用于在interceptor(拦截器)之前,不像interceptor一样依赖于springmvc框架,只需要依赖于serverlet。
- 比较原始的一种方法是定义一个类实现javax.servlet.Filter接口
Expand Down
4 changes: 2 additions & 2 deletions src/main/java/com/four/SOURCECODE.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(beanFactory, g
- AutoConfigurationImportSelector 加载配置,先看Exclude注解有没有需要过滤的,在执行过滤器过滤出需要加载的autoConfiguration
## Spring transaction
![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/four/springTransaction.jpg)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/four/springTransaction.jpg)
Spring 框架中,事务管理相关最重要的 3 个接口如下:
- PlatformTransactionManager: (平台)事务管理器,Spring 事务策略的核心。
Expand Down Expand Up @@ -351,5 +351,5 @@ public Object invoke(Object proxy, Method method, Object[] args) throws Throwabl
```
### AspectJ的方法织入
AspectJ的Aop 在方法调用的时候添加了AOP对应的拦截方法,根据对应的拦截类型
![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/four/methodAop.jpg)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/four/methodAop.jpg)
6 changes: 3 additions & 3 deletions src/main/java/com/four/SPRING-CLOUD.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,13 @@
相关资料: [spring-cloud-openFeign源码深度](https://blog.csdn.net/sinat_29899265/article/details/86577997)

![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/four/OpenFeignStructure.png)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/four/OpenFeignStructure.png)


## Hystrix

整体流程图
![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/four/hystrix-command-flow-chart.png)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/four/hystrix-command-flow-chart.png)

基于注解的实现流程:
1. 使用HystrixCommandAspect切面实现,通过扫描注解属性进而新建不同的HystrixCommand对象。
Expand Down Expand Up @@ -53,7 +53,7 @@ public HystrixInvokable create(MetaHolder metaHolder) {


### 断路器流程
![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/four/circuit-breaker.png)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/four/circuit-breaker.png)


```
Expand Down
4 changes: 2 additions & 2 deletions src/main/java/com/four/SPRING-SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

[Spring Security官网][https://docs.spring.io/spring-security/site/docs/5.4.2/reference/html5/#servlet-architecture]

![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/four/spring-security.jpg)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/four/spring-security.jpg)


## 认证 Authentication
Expand Down Expand Up @@ -122,7 +122,7 @@ for (Map.Entry<String, String> entry : resourcePermissions.entrySet()) {
进行Session 认证的时候,我们一般使用 **Cookie 来存储 SessionId**,当我们登陆后后端生成一个SessionId放在Cookie中返回给客户端,服务端通过Redis或者其他存储工具记录保存着这个Sessionid,客户端登录以后每次请求都会带上这个SessionId,服务端通过这个SessionId来标示你这个人。如果别人通过 cookie拿到了 SessionId 后就可以代替你的身份访问系统了。


![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/four/Cookies-attack.jpg)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/four/Cookies-attack.jpg)

可以使用token认证的方式避免误点攻击链接导致的跨服务请求问题。
> 基于token 认证经常将认证凭证存储在local storage中,在请求的时候前端再动态添加凭证到请求中。因此误点的外部链接无法添加token到转发的请求中。
Expand Down
6 changes: 3 additions & 3 deletions src/main/java/com/four/SPRING.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ Spring AOP 属于运行时增强,而 AspectJ 是编译时增强。 Spring AOP

## Spring 循环依赖
对于普通的循环依赖如A 依赖B, B依赖A。在初始化A的时候,会实例化B,实例化B发现需要A的引用,这时候通过缓存返回A的引用。虽然A还未初始化完毕,但是由于是对象的引用,所以最终初始化完成的时候,两个对象均是初始化完整的。
![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/four/iocAutowire.png)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/four/iocAutowire.png)

getSingleton(beanName, true)这个方法实际上就是到缓存中尝试去获取Bean,整个缓存分为三级
- singletonObjects,一级缓存,存储的是所有创建好了的单例Bean
Expand All @@ -107,10 +107,10 @@ getSingleton(beanName, true)这个方法实际上就是到缓存中尝试去获
> 这个工厂的目的在于**延迟对实例化阶段生成对象的代理**,只有真正发生循环依赖的时候,才去提前生成代理对象,否则只会创建一个工厂并将其放入到三级缓存中,但是不会去通过这个工厂去真正创建对象。
三级缓存引入对于对象延迟实例化的体现。
![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/four/iocAutowire.jpg)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/four/iocAutowire.jpg)

不同注入方式对于循环引用的影响
![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/four/iocAutowire3.jpg)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/four/iocAutowire3.jpg)

### 相关文章
[讲一讲Spring中的循环依赖](https://mp.weixin.qq.com/s/kS0K5P4FdF3v-fiIjGIvvQ)
Expand Down
6 changes: 3 additions & 3 deletions src/main/java/com/four/SPRINGBOOT.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# Spring boot

## Spring boot start
![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/four/springBootLoadother.png)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/four/springBootLoadother.png)

![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/four/spingLoad.jpg)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/four/spingLoad.jpg)

## spring 构造方法
```
Expand Down Expand Up @@ -219,7 +219,7 @@ public void refresh() throws BeansException, IllegalStateException {
---

## spring boot相关知识
![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/four/BootStrapLoad.png)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/four/BootStrapLoad.png)
Spring Cloud 构建于 Spring Boot 之上,在 Spring Boot 中有两种上下文,一种是 bootstrap,另外一种是 application,
- application 配置文件这个容易理解,主要用于 Spring Boot 项目的自动化配置。
- bootstrap 是应用程序的父上下文,也就是说 bootstrap 加载优先于 applicaton。
Expand Down
4 changes: 2 additions & 2 deletions src/main/java/com/learning/algorithm/ALGORITHM.md
Original file line number Diff line number Diff line change
Expand Up @@ -755,7 +755,7 @@ public class Solution {
## 回溯
回溯法⼀般是在集合中递归搜索,集合的⼤⼩构成了树的宽度,递归的深度构成的树的深度。

![image](https://github.com/rbmonster/file-storage/blob/main/learning-note/learning/basic/backTracking.png)
![image](https://gitee.com/rbmon/file-storage/raw/main/learning-note/learning/basic/backTracking.png)

### 伪代码模版
```
Expand Down Expand Up @@ -816,7 +816,7 @@ for (选择:本层集合中元素(树中节点孩⼦的数量就是集合的
组合问题可以抽象为树形结构,那么“使⽤过”在这个树形结构上是有两个维度的,⼀个维度是同⼀树枝上“使⽤过”,⼀个维度是同⼀树层上“使⽤过”。\
常规使用树层去重,树枝去重会导致过多无谓的查找,而树层去重对于无用的查找可以及时的中断break

![image](https://github.com/rbmonster/file-storage/blob/main/learning-note/learning/basic/backTrackingDuplicate.png)
![image](https://gitee.com/rbmon/file-storage/raw/main/learning-note/learning/basic/backTrackingDuplicate.png)


树层去重
Expand Down
2 changes: 1 addition & 1 deletion src/main/java/com/learning/basic/COLLECTION.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# 集合
![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/learning/basic/collectionfamily.jpg)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/learning/basic/collectionfamily.jpg)
- 集合主要分为两大类,一个实现collection接口的,一个是实现了Map接口的。

## List
Expand Down
12 changes: 6 additions & 6 deletions src/main/java/com/learning/basic/CONCURRENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
定义: 一个线程对共享变量的修改,另外一个线程能够立刻看到,我们称为可见性

Java 内存模型规定,将所有的变量都存放在 主内存中,当线程使用变量时,会把主内存里面的变量 复制 到自己的工作空间或者叫作 私有内存
![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/learning/basic/workingmemory.jpg)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/learning/basic/workingmemory.jpg)

刷新的场景:
>1. 主内存中有变量 x,初始值为 0
Expand Down Expand Up @@ -86,7 +86,7 @@ count++ 分解为四步,解释一下字节码的指令,


## java内存模型
![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/learning/jvm/java-memory.jpg)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/learning/jvm/java-memory.jpg)

java内存模型是关注在虚拟机中把变量值存储到内存和从内存取出变量这样的底层细节。
> 此处的变量包括了实例字段、静态字段和构成数组对象的元素,但是不包括局部变量和方法参数。 \
Expand Down Expand Up @@ -214,7 +214,7 @@ publicclass SynchronizedDemo {
javap -c -s -v -l SynchronizedDemo.class 反编译
```
![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/learning/basic/synchronizeCode.jpg)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/learning/basic/synchronizeCode.jpg)
synchronized 同步语句块的实现使用的是 monitorenter 和 monitorexit 指令,其中 monitorenter 指令指向同步代码块的开始位置,monitorexit 指令则指明同步代码块的结束位置。\
当计数器为0则可以成功获取,获取后将锁计数器设为1也就是加1。相应的在执行 monitorexit 指令后,将锁计数器设为0,表明锁被释放。

Expand All @@ -226,11 +226,11 @@ public class SynchronizedDemo2 {
}
}
```
![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/learning/basic/synchronizeMethod.jpg)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/learning/basic/synchronizeMethod.jpg)
方法体出现ACC_SYNCHRONIZED 标识,该标识指明了该方法是一个同步方法,JVM 通过该 ACC_SYNCHRONIZED 访问标志来辨别一个方法是否声明为同步方法,从而执行相应的同步调用。

### 对象创建与加锁
![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/learning/basic/objectCreate.png)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/learning/basic/objectCreate.png)

一个对象在new出来之后在内存中主要分为4个部分:
1. MarkWord这部分其实就是加锁的核心,同时还包含的对象的一些生命信息,例如是否GC、经过了几次Young GC还存活。
Expand Down Expand Up @@ -310,7 +310,7 @@ public class SynchronizedDemo2 {
重量级锁:阻塞所有等待竞争的线程,防止CPU空转,阻塞等待线程1释放锁后进入无锁状态重新竞争。

##### 工作原理
![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/learning/basic/synchronizeHeavyLock.jpg)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/learning/basic/synchronizeHeavyLock.jpg)

```
ContentionList:所有请求锁的线程将被首先放在该竞争队列。
Expand Down
6 changes: 3 additions & 3 deletions src/main/java/com/learning/basic/CONCURRENTTOOL.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ tryReleaseShared(int)//共享方式。尝试释放资源,成功则返回true

#### AQS 相关方法处理说明
##### 获取锁的框架方法
![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/learning/basic/aqsblocklock.jpg)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/learning/basic/aqsblocklock.jpg)
```
// 尝试获取锁
public final void acquire(int arg) {
Expand Down Expand Up @@ -133,7 +133,7 @@ unparkSuccessor:资源释放后的队列抢资源逻辑


#### 条件队列
![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/learning/basic/conditionqueue.jpg)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/learning/basic/conditionqueue.jpg)
condition:条件队列的实现,常可以用在生产者-消费者的场景中。在所对象中内置一个newCondition方法,用于创建一个条件队列。\
condition 的方法主要就两个await等待、signal唤醒

Expand Down Expand Up @@ -359,7 +359,7 @@ countdown方法:CAS+自旋扣减statue状态。当状态为0时,唤醒await
内部使用ReentrantLock 非公平锁,每次await时,加锁扣减数量,使用condition的await等待唤醒。CountDownLatch 基于 AQS 的共享模式的使用,而 CyclicBarrier 基于 Condition 来实现。
> 数量扣减为0时,如果有定义栅栏开始的方法则执行,并调用condition的signAll,条件单链表逐个唤醒。 generation 代表栅栏重复使用的一代或者一个周期。

![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/learning/basic/cyclicBarrier.jpg)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/learning/basic/cyclicBarrier.jpg)

什么时候栅栏会被打破,总结如下:
- 中断,如果某个等待的线程发生了中断,那么会打破栅栏,同时抛出 InterruptedException 异常;
Expand Down
2 changes: 1 addition & 1 deletion src/main/java/com/learning/basic/JAVA_BASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -546,7 +546,7 @@ java.lang.annotation 提供了四种元注解,专门注解其他的注解(

## 线程
### 线程状态
![avatar](https://github.com/rbmonster/file-storage/blob/main/learning-note/learning/concurrent/threadState.jpg)
![avatar](https://gitee.com/rbmon/file-storage/raw/main/learning-note/learning/concurrent/threadState.jpg)

1. 新建(NEW):创建后尚未启动。
2. 可运行(RUNABLE):正在 Java 虚拟机中运行。但是在操作系统层面,它可能处于运行状态,也可能等待资源调度(例如处理器资源),资源调度完成就进入运行状态。所以该状态的可运行是指可以被运行,具体有没有运行要看底层操作系统的资源调度。
Expand Down
Loading

0 comments on commit 1aebdfb

Please sign in to comment.