1. 准备solr环境
过程略,假设已有了solr环境,地址:
http://192.168.75.101:8983/solr/
Schema中添加索引:
2. 创建一个springMVC工程
起名为:SpringMVC-Solr
创建UserController,简单的返回一个页面:
@RequestMapping("/login")
public String login(){
return "login";
}
3. 导入solr相关的包
4. 测试代码:
@Test
public void testQuery(){
String queryStr = "*:*";
SolrQuery query = new SolrQuery(queryStr);
SolrDocumentList sdls = getResults(query);
System.out.println(sdls.toString());
}
public SolrDocumentList getResults(SolrQuery query) {
QueryResponse response = null;
HttpSolrClient solrClient = SolrUtil.getSolrClient("http://192.168.75.101:8983/solr/harvewifi", "", "");
try {
response = solrClient.query(query);
if (response == null) {
return null;
}
} catch (SolrServerException | IOException e) {
e.printStackTrace();
}
return response.getResults();
}
public final class SolrUtil {
public static HttpSolrClient getSolrClient(String uri, String username, String password) {
if (Strings.isNullOrEmpty(username)) {
return new HttpSolrClient(uri);
} else {
return new HttpSolrClient(uri, createHttpClient(uri, username, password));
}
}
private static HttpClient createHttpClient(String uri, String username, String password) {
final URI scopeUri = URI.create(uri);
final BasicCredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(new AuthScope(scopeUri.getHost(), scopeUri.getPort()), new UsernamePasswordCredentials(username, password));
final HttpClientBuilder builder = HttpClientBuilder.create().setMaxConnTotal(128).setMaxConnPerRoute(32).setDefaultRequestConfig(RequestConfig.copy(RequestConfig.DEFAULT).setRedirectsEnabled(true).build()).setDefaultCredentialsProvider(credentialsProvider).addInterceptorFirst(new PreemptiveAuthInterceptor());
return builder.build();
}
}
输出:
{numFound=5,start=0,docs=[
SolrDocument{id=change.me, title=[change.me], _version_=1558755453827022848}, SolrDocument{id=0001, user_name=gary, _version_=1558756647820591104}, SolrDocument{id=c0002, user_name=商品名称2, _version_=1558757289744138240}, SolrDocument{id=001, user_uuid=001, user_name=张三, user_age=23, user_birthday=1988-10-24 12:40:21, user_city=北京, _version_=1558767452121202688}, SolrDocument{user_name=李四, user_age=24, id=002, user_city=北京, user_birthday=1988-10-24 11:40:21, user_uuid=002, _version_=1558768367416901632}]
}
可见查出了数据。
5. 使用配置文件
上面是在方法中写死的solr地址,需要使用配置文件,在根目录创建resource文件夹,拷贝一个属性文件过去,启名为solr.properties
内容为:
solr.path=http://localhost:8983/solr
solr.username=
solr.password=
同时在build path中将resource目录添加进去。
在spring的配置文件中添加配置:
<bean id="appProperty" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<array>
<value>classpath:solr.properties</value>
</array>
</property>
</bean>
代码中就可以直接通过@Value注解获取了
@Value("${solr.path}")
private String solrUrl;
@Value("${solr.username}")
private String solrUserName;
@Value("${solr.password}")
private String solrPassword;
使用时很简单了:
HttpSolrClient solrClient = SolrUtil.getSolrClient(solrUrl, solrUserName, solrPassword);
response = solrClient.query(query);
6. 使用注解
上面的例子中其实是通过@Value注解获取配置信息,再通过util的方法去new一个HttpSolrClient
在使用的时候还是不太方便。
另一种更好的方法是,添加一个配置类:
在此类中通过@value获取配置信息,并添加一个@bean的注解,在此方法中去new一个httpSolrClient。
@Configuration
public class SolrTemplateConfig {
@Value("${solr.path}")
private String uri;
@Value("${solr.username}")
private String username;
@Value("${solr.password}")
private String password;
@Resource(name = "solr.httpClientBuilder")
private HttpClientBuilder solrHttpClientBuilder;
@Bean
public HttpSolrClient httpSolrClient() {
if (Strings.isNullOrEmpty(username)) {
return new HttpSolrClient(uri);
}
final URI scopeUri = URI.create(uri);
final BasicCredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(new AuthScope(scopeUri.getHost(), scopeUri.getPort()), new UsernamePasswordCredentials(username, password));
solrHttpClientBuilder.setMaxConnTotal(128).setMaxConnPerRoute(32).setDefaultRequestConfig(RequestConfig.copy(RequestConfig.DEFAULT).setRedirectsEnabled(true).build()).setDefaultCredentialsProvider(credentialsProvider).addInterceptorFirst(new PreemptiveAuthInterceptor());
HttpSolrClient client = new HttpSolrClient(uri, solrHttpClientBuilder.build());
return client;
}
private static class PreemptiveAuthInterceptor implements HttpRequestInterceptor {
@Override
public void process(final HttpRequest request, final HttpContext context) throws HttpException, IOException {
final AuthState authState = (AuthState) context.getAttribute(HttpClientContext.TARGET_AUTH_STATE);
if (authState.getAuthScheme() == null) {
final CredentialsProvider credsProvider = (CredentialsProvider) context.getAttribute(HttpClientContext.CREDS_PROVIDER);
final HttpHost targetHost = (HttpHost) context.getAttribute(HttpCoreContext.HTTP_TARGET_HOST);
final Credentials creds = credsProvider.getCredentials(new AuthScope(targetHost.getHostName(), targetHost.getPort()));
if (creds == null) {
throw new HttpException("No creds provided for preemptive auth.");
}
authState.update(new BasicScheme(), creds);
}
}
}
}
在使用的地方就方便了,直接使用:
@Resource
HttpSolrClient httpSolrClient;
就可以用了。
但前提是在spring的配置文件中,添加配置:
<!-- httpclient的实现,带连接池 -->
<bean id="ky.pollingConnectionManager"
class="org.apache.http.impl.conn.PoolingHttpClientConnectionManager">
<!--整个连接池的并发 -->
<property name="maxTotal" value="1000" />
<!--每个主机的并发 -->
<property name="defaultMaxPerRoute" value="1000" />
</bean>
<bean id="solr.httpClientBuilder" class="org.apache.http.impl.client.HttpClientBuilder"
factory-method="create">
<property name="connectionManager" ref="ky.pollingConnectionManager" />
<property name="maxConnTotal" value="128" />
<property name="maxConnPerRoute" value="32" />
<property name="retryHandler">
<bean class="org.apache.http.impl.client.DefaultHttpRequestRetryHandler">
<constructor-arg value="2" />
<constructor-arg value="true" />
</bean>
</property>
<property name="defaultHeaders">
<list>
<bean class="org.apache.http.message.BasicHeader">
<constructor-arg value="User-Agent"/>
<constructor-arg value="Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.16 Safari/537.36"/>
</bean>
<bean class="org.apache.http.message.BasicHeader">
<constructor-arg value="Accept-Encoding"/>
<constructor-arg value="gzip,deflate"/>
</bean>
<bean class="org.apache.http.message.BasicHeader">
<constructor-arg value="Accept-Language"/>
<constructor-arg value="zh-CN"/>
</bean>
</list>
</property>
</bean>
而且还需要扫描config包:
<context:component-scan base-package="com.gary.config"></context:component-scan>
7. 添加数据
添加一个实体类:User:
public class User implements Serializable{
@Field("user_name")
private String name;
@Field("user_age")
private Integer age;
@Field("id")
private String id;
@Field("user_city")
private String city;
@Field("user_birthday")
private String birthday;
@Field("user_uuid")
private String userId;
...
}
controller中:
@Controller
public class UserController {
@Autowired
private UserService userService;
@RequestMapping("/showUsers")
public String showUsers(){
List<User> list = userService.query();
if (list != null){
System.out.println(list.toString());
}
return "userList";
}
@RequestMapping("/addUser")
public String addUser(){
User user = new User("李四",24,"002","北京","1988-10-24 11:40:21");
userService.createUser(user);
System.out.println("addUser");
return "addUserSuccess";
}
}
Service中:
@Service
public class UserService {
@Autowired
private UserDaoImpl userDao;
public User createUser(User user){
return userDao.createUser2(user);
}
public List<User> query(){
return userDao.query();
}
}
Dao中:
@Repository
public class UserDaoImpl {
@Resource
private SolrDao solrDao;
public List<User> query(){
List<User> Userlist = new ArrayList<User>();
//...
return Userlist;
}
public User createUser1(User user){
if (!solrDao.addBean(user)){
return null;
}
return user;
}
public User createUser2(User user){
SolrInputDocument document = new SolrInputDocument();
document.addField("id", user.getId());
document.addField("user_uuid", user.getId());
document.addField("user_name", user.getName());
document.addField("user_age", user.getAge());
document.addField("user_birthday", user.getBirthday());
document.addField("user_city", user.getCity());
//把document对象添加到索引库中
solrDao.addDocument(document);
return user;
}
}
@Repository
public class SolrDaoImpl implements SolrDao {
@Resource
private HttpSolrClient httpSolrClient;
public SolrDaoImpl() {
}
@Override
public SolrDocumentList getResults(SolrQuery query) {
QueryResponse response = null;
try {
response = httpSolrClient.query(query);
if (response == null) {
return null;
}
} catch (SolrServerException | IOException e) {
e.printStackTrace();
} finally {
}
return response.getResults();
}
@Override
public boolean addDocument(SolrInputDocument solrInputDocument) {
try{
httpSolrClient.add(solrInputDocument);
//提交修改
httpSolrClient.commit();
return true;
}
catch (Exception e){
e.printStackTrace();
}
return false;
}
@Override
public boolean addBean(Object obj) {
try{
httpSolrClient.addBean(obj);
//提交修改
httpSolrClient.commit();
return true;
}
catch (Exception e){
e.printStackTrace();
}
return false;
}
}
测试:
http://localhost:8080/SpringMVC-Solr/addUser
上面addDocument和addBean都可以成功添加记录。
8. 修改数据
同添加数据,是通过id来判断是否为修改。
注意,它会先把id相同的记录清空,再添加。因此不能只修改某一个字段。
9. 删除数据
@Override
public boolean deleteById(String id){
try {
httpSolrClient.deleteById(id);
httpSolrClient.commit();
} catch (SolrServerException | IOException e) {
e.printStackTrace();
return false;
}
return true;
}
@Override
public boolean deleteByQuery(String query){
try {
httpSolrClient.deleteByQuery(query);
httpSolrClient.commit();
} catch (SolrServerException | IOException e) {
e.printStackTrace();
return false;
}
return true;
}
10. Commit方式
solr提交文档的两种方式。
1. commit
一般提交又或者叫硬提交(hard commit), 使用这种提交会把文档立即持久化到磁盘,并可以让你能立马查询到它,因为它会开启一个新的searcher,但是它缺点很明显,就是很耗性能,并会阻塞到提交任务完成,使用它是非常昂贵的操作。
在提交的文档的url里使用commit=true即可使用。
2. softcommit
软提交,这种提交不会立即把数据写到磁盘,但它可以使你能立即查询到它,就是所谓的支持near real-time (NRT) searching近实时查询,而且这样的操作不昂贵。
通过对这两种提交方式的了解,我们知道了软硬提交,除了在客户端显示设置这两种提交,我们可以在solrconfig.xml里配置这两种提交方式自动完成。
<autoCommit>
<maxTime>${solr.autoCommit.maxTime:30000}</maxTime>
<openSearcher>false</openSearcher>
</autoCommit>
<autoSoftCommit>
<maxTime>${solr.autoSoftCommit.maxTime:5000}</maxTime>
</autoSoftCommit>
在autoCommit里设置了文档每隔30000ms就去做一次硬提交,openSearcher属性设置是否每一次硬提交都开启一个新的Searcher,由于开启Searcher也是昂贵的操作,推荐不要开启,可以通过配合下面的autoSoftCommit来实现快速检索到提交的问题。
autoSoftCommit里设置了5000ms做一个软提交。
这样我们在客户端就可以不维护提交的方式和时间了,solr会自动来做这些事情。
Solr的softCommit是Write-ahead Logging的,所以不必担心softCommit的数据会丢失。Log数据就在$solrHome/collection/data/tlog/下。
对应代码:
solrTemplate.saveBean(object);
solrTemplate.softCommit();
//solrTemplate.commit();