zoukankan      html  css  js  c++  java
  • SpringBoot-MongoDB 索引冲突分析及解决

    一、背景

    spring-data-mongo 实现了基于 MongoDB 的 ORM-Mapping 能力,
    通过一些简单的注解、Query封装以及工具类,就可以通过对象操作来实现集合、文档的增删改查;
    在 SpringBoot 体系中,spring-data-mongo 是 MongoDB Java 工具库的不二之选。

    二、问题产生

    在一次项目问题的追踪中,发现SpringBoot 应用启动失败,报错信息如下:

    Error creating bean with name 'mongoTemplate' defined in class path resource [org/bootfoo/BootConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.data.mongodb.core.MongoTemplate]: Factory method 'mongoTemplate' threw exception; nested exception is org.springframework.dao.DataIntegrityViolationException: Cannot create index for 'deviceId' in collection 'T_MDevice' with keys '{ "deviceId" : 1}' and options '{ "name" : "deviceId"}'. Index already defined as '{ "v" : 1 , "unique" : true , "key" : { "deviceId" : 1} , "name" : "deviceId" , "ns" : "appdb.T_MDevice"}'.; nested exception is com.mongodb.MongoCommandException: Command failed with error 85: 'exception: Index with name: deviceId already exists with different options' on server 127.0.0.1:27017. The full response is { "createdCollectionAutomatically" : false, "numIndexesBefore" : 6, "errmsg" : "exception: Index with name: deviceId already exists with different options", "code" : 85, "ok" : 0.0 }
    	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:588)
    	at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:88)
    	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:366)
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1264)
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553)
    
    ...
    
    Caused by: org.springframework.dao.DataIntegrityViolationException: Cannot create index for 'deviceId' in collection 'T_MDevice' with keys '{ "deviceId" : 1}' and options '{ "name" : "deviceId"}'. Index already defined as '{ "v" : 1 , "unique" : true , "key" : { "deviceId" : 1} , "name" : "deviceId" , "ns" : "appdb.T_MDevice"}'.; nested exception is com.mongodb.MongoCommandException: Command failed with error 85: 'exception: Index with name: deviceId already exists with different options' on server 127.0.0.1:27017. The full response is { "createdCollectionAutomatically" : false, "numIndexesBefore" : 6, "errmsg" : "exception: Index with name: deviceId already exists with different options", "code" : 85, "ok" : 0.0 }
    	at org.springframework.data.mongodb.core.index.MongoPersistentEntityIndexCreator.createIndex(MongoPersistentEntityIndexCreator.java:157)
    	at org.springframework.data.mongodb.core.index.MongoPersistentEntityIndexCreator.checkForAndCreateIndexes(MongoPersistentEntityIndexCreator.java:133)
    	at org.springframework.data.mongodb.core.index.MongoPersistentEntityIndexCreator.checkForIndexes(MongoPersistentEntityIndexCreator.java:125)
    	at org.springframework.data.mongodb.core.index.MongoPersistentEntityIndexCreator.<init>(MongoPersistentEntityIndexCreator.java:91)
    	at org.springframework.data.mongodb.core.index.MongoPersistentEntityIndexCreator.<init>(MongoPersistentEntityIndexCreator.java:68)
    	at org.springframework.data.mongodb.core.MongoTemplate.<init>(MongoTemplate.java:229)
    	at org.bootfoo.BootConfiguration.mongoTemplate(BootConfiguration.java:121)
    	at org.bootfoo.BootConfiguration$$EnhancerBySpringCGLIB$$1963a75.CGLIB$mongoTemplate$2(<generated>)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    	at java.lang.reflect.Method.invoke(Unknown Source)
    	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:162)
    	... 58 more
    
    Caused by: com.mongodb.MongoCommandException: Command failed with error 85: 'exception: Index with name: deviceId already exists with different options' on server 127.0.0.1:27017. The full response is { "createdCollectionAutomatically" : false, "numIndexesBefore" : 6, "errmsg" : "exception: Index with name: deviceId already exists with different options", "code" : 85, "ok" : 0.0 }
    	at com.mongodb.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:115)
    	at com.mongodb.connection.CommandProtocol.execute(CommandProtocol.java:114)
    	at com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:168)
    
    
    
    

    关键信息:org.springframework.dao.DataIntegrityViolationException: Cannot create index

    从异常信息上看,出现的是索引冲突(Command failed with error 85),spring-data-mongo 组件在程序启动时会实现根据注解创建索引的功能。
    查看业务实体定义:

    @Document(collection = "T_MDevice")
    public class MDevice {
    
        @Id
        private String id;
    
        @Indexed(unique=true)
        private String deviceId;
    
    

    deviceId 这个字段上定义了一个索引,unique=true表示这是一个唯一索引。
    我们继续 查看 MongoDB中表的定义:

    db.getCollection('T_MDevice').getIndexes()
    
    >>
    [
        {
            "v" : 1,
            "key" : {
                "_id" : 1
            },
            "name" : "_id_",
            "ns" : "appdb.T_MDevice"
        },
        {
            "v" : 1,
            "key" : {
                "deviceId" : 1
            },
            "name" : "deviceId",
            "ns" : "appdb.T_MDevice"
        }
    ]
    
    

    发现数据库表中同样存在一个名为 deviceId的索引,但是并非唯一索引!

    三、详细分析

    为了核实错误产生的原因,我们尝试通过 Mongo Shell去执行索引的创建,发现返回了同样的错误。
    通过将数据库中的索引删除,或更正为 unique=true 之后可以解决当前的问题。

    从严谨度上看,一个索引冲突导致 SpringBoot 服务启动不了,是可以接受的。
    但从灵活性来看,是否有某些方式能禁用索引的自动创建,或者仅仅是打印日志呢?

    尝试 google spring data mongodb disable index creation
    发现 JIRA-DATAMONGO-1201在2015年就已经提出,至今未解决。

    stackoverflow 找到许多同样问题
    但大多数的解答是不采用索引注解,选择其他方式对索引进行管理。

    这些结果并不能令人满意。

    尝试查看 spring-data-mongo 的机制,定位到 MongoPersistentEntityIndexCreator类:

    1. 初始化方法中,会根据 MappingContext(实体映射上下文)中已有的实体去创建索引
    public MongoPersistentEntityIndexCreator(MongoMappingContext mappingContext, MongoDbFactory mongoDbFactory,
    			IndexResolver indexResolver) {
            ...
            //根据已有实体创建
    		for (MongoPersistentEntity<?> entity : mappingContext.getPersistentEntities()) {
    			checkForIndexes(entity);
    		}
    	}
    
    
    1. 在接收到MappingContextEvent时,创建对应实体的索引
    	public void onApplicationEvent(MappingContextEvent<?, ?> event) {
    
    		if (!event.wasEmittedBy(mappingContext)) {
    			return;
    		}
    
    		PersistentEntity<?, ?> entity = event.getPersistentEntity();
    
    		// Double check type as Spring infrastructure does not consider nested generics
    		if (entity instanceof MongoPersistentEntity) {
                //创建单个实体索引
    			checkForIndexes((MongoPersistentEntity<?>) entity);
    		}
    	}
    
    

    MongoPersistentEntityIndexCreator是通过MongoTemplate引入的,如下:

    	public MongoTemplate(MongoDbFactory mongoDbFactory, MongoConverter mongoConverter) {
    
    		Assert.notNull(mongoDbFactory);
    
    		this.mongoDbFactory = mongoDbFactory;
    		this.exceptionTranslator = mongoDbFactory.getExceptionTranslator();
    		this.mongoConverter = mongoConverter == null ? getDefaultMongoConverter(mongoDbFactory) : mongoConverter;
    		...
    
    		// We always have a mapping context in the converter, whether it's a simple one or not
    		mappingContext = this.mongoConverter.getMappingContext();
    		// We create indexes based on mapping events
    		if (null != mappingContext && mappingContext instanceof MongoMappingContext) {
    			indexCreator = new MongoPersistentEntityIndexCreator((MongoMappingContext) mappingContext, mongoDbFactory);
    			eventPublisher = new MongoMappingEventPublisher(indexCreator);
    			if (mappingContext instanceof ApplicationEventPublisherAware) {
    				((ApplicationEventPublisherAware) mappingContext).setApplicationEventPublisher(eventPublisher);
    			}
    		}
    	}
    
    
        ...
        //MongoTemplate实现了 ApplicationContextAware,当ApplicationContext被实例化时被感知
    	public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
    
    		prepareIndexCreator(applicationContext);
    
    		eventPublisher = applicationContext;
    		if (mappingContext instanceof ApplicationEventPublisherAware) {
                //MappingContext作为事件来源,向ApplicationContext发布
    			((ApplicationEventPublisherAware) mappingContext).setApplicationEventPublisher(eventPublisher);
    		}
    		resourceLoader = applicationContext;
    	}
    
        ...
        //注入事件监听
    	private void prepareIndexCreator(ApplicationContext context) {
    
    		String[] indexCreators = context.getBeanNamesForType(MongoPersistentEntityIndexCreator.class);
    
    		for (String creator : indexCreators) {
    			MongoPersistentEntityIndexCreator creatorBean = context.getBean(creator, MongoPersistentEntityIndexCreator.class);
    			if (creatorBean.isIndexCreatorFor(mappingContext)) {
    				return;
    			}
    		}
    
    		if (context instanceof ConfigurableApplicationContext) {
                //使 IndexCreator 监听 ApplicationContext的事件
    			((ConfigurableApplicationContext) context).addApplicationListener(indexCreator);
    		}
    	}
    
    

    由此可见,MongoTemplate在初始化时,先通过MongoConverter 带入 MongoMappingContext,
    随后完成一系列初始化,整个过程如下:

    • 实例化 MongoTemplate;
    • 实例化 MongoConverter;
    • 实例化 MongoPersistentEntityIndexCreator;
    • 初始化索引(通过MappingContext已有实体);
    • Repository初始化 -> MappingContext 发布映射事件;
    • ApplicationContext 将事件通知到 IndexCreator;
    • IndexCreator 创建索引

    在实例化过程中,没有任何配置可以阻止索引的创建。

    四、解决问题

    从前面的分析中,可以发现问题关键在 IndexCreator,能否提供一个自定义的实现呢,答案是可以的!

    实现的要点如下

    1. 实现一个IndexCreator,可继承MongoPersistentEntityIndexCreator,去掉索引的创建功能;
    2. 实例化 MongoConverter和 MongoTemplate时,使用一个空的 MongoMappingContext对象避免初始化索引;
    3. 将自定义的IndexCreator作为Bean进行注册,这样在prepareIndexCreator方法执行时,
      原来的 MongoPersistentEntityIndexCreator不会监听ApplicationContext的事件
    4. IndexCreator 实现了ApplicationContext监听,接管 MappingEvent事件处理。

    实例化Bean

        @Bean
        public MongoMappingContext mappingContext() {
            return new MongoMappingContext();
        }
    
        // 使用 MappingContext 实例化 MongoTemplate
        @Bean
        public MongoTemplate mongoTemplate(MongoDbFactory mongoDbFactory, MongoMappingContext mappingContext) {
            MappingMongoConverter converter = new MappingMongoConverter(new DefaultDbRefResolver(mongoDbFactory),
                    mappingContext);
            converter.setTypeMapper(new DefaultMongoTypeMapper(null));
    
            MongoTemplate mongoTemplate = new MongoTemplate(mongoDbFactory, converter);
    
            return mongoTemplate;
        }
    
    

    自定义IndexCreator

        // 自定义IndexCreator实现
        @Component
        public static class CustomIndexCreator extends MongoPersistentEntityIndexCreator {
    
            // 构造器引用MappingContext
            public CustomIndexCreator(MongoMappingContext mappingContext, MongoDbFactory mongoDbFactory) {
                super(mappingContext, mongoDbFactory);
            }
    
            public void onApplicationEvent(MappingContextEvent<?, ?> event) {
                PersistentEntity<?, ?> entity = event.getPersistentEntity();
    
                // 获得Mongo实体类
                if (entity instanceof MongoPersistentEntity) {
                    System.out.println("Detected MongoEntity " + entity.getName());
                    
                    //可实现索引处理..
                }
            }
        }
    
    

    在这里 CustomIndexCreator继承了MongoPersistentEntityIndexCreator,将自动接管MappingContextEvent事件的监听。
    在业务实现上可以根据需要完成索引的处理!

    小结

    spring-data-mongo 提供了非常大的便利性,但在灵活性支持上仍然不足。上述的方法实际上有些隐晦,在官方文档中并未提及这样的方式。
    ORM-Mapping 框架在实现Schema映射处理时需要考虑校验级别,比如 Hibernate便提供了 none/create/update/validation 多种选择,毕竟这对开发者来说更加友好。
    期待 spring-data-mongo 在后续的演进中能尽快完善 Schema的管理功能!

  • 相关阅读:
    BZOJ 1391: [Ceoi2008]order
    BZOJ 4504: K个串
    2019 年百度之星·程序设计大赛
    POJ 2398 Toy Storage (二分 叉积)
    POJ 2318 TOYS (二分 叉积)
    HDU 6697 Closest Pair of Segments (计算几何 暴力)
    HDU 6695 Welcome Party (贪心)
    HDU 6693 Valentine's Day (概率)
    HDU 6590 Code (判断凸包相交)
    POJ 3805 Separate Points (判断凸包相交)
  • 原文地址:https://www.cnblogs.com/littleatp/p/10043447.html
Copyright © 2011-2022 走看看