sinat_34080511-CSDN博客

原创 mac vscode 安装gdb

2024-04-23 18:42:27 208

# https://stackoverflow.com/questions/18424228/cosine-similarity-between-2-number-lists# a = [3, 45, 7, 2]# b = [2, 54, 13, 15]# a = torch.Tensor([1, 2, 3])# F.cosine_similarity(a,b, dim=0)# torch.cosine_similarity(a, b,dim=0)# print 1 - spatial.dis.

2022-03-02 11:42:59 196

原创 python3.7 pyspider安装

修改两个地方，1. async关键字造成的错误，把下面代码中的async关键字替换python3.7/site-packages/pyspider/run.pypython3.7/site-packages/pyspider/fetcher/tornado_fetcher.pypython3.7/site-packages/pyspider/webui/app.py2.werkzeug版本错误pip uninstall werkzeugpip install werkzeu...

2021-12-20 15:08:17 529

原创升级gcc

转载输入 yum list | grep gcc 看是否有devtoolset-7-gcc、devtoolset-7-gcc-c++.x86_64 等依赖包可供选择。如果没有，进入步骤2，否则进入步骤3。你可能需要更换镜像源或者将原来的yum卸载，更换为非centos自带的yum。我推荐用后者，因为若只更换镜像源，yum还是无法安装gcc、g++等工具，不方便。在这里，我推荐按照https://blog.csdn.net/jianm_liu/article/details/78316690 这篇博文

2021-12-18 21:45:00 2592

原创 cosine similarity contrastive loss

TwinBERT:logistic regression layer is applied to the cosine similarity score and convert it to [0, 1].TwinBERT: Distilling Knowledge to Twin-Structured BERT Models for Efficient Retrieval https://arxiv.org/pdf/2002.06275.pdfLearning Text Similarity wi.

2021-12-15 18:54:02 2066

原创 c++time

auto start = std::chrono::high_resolution_clock::now(); for (auto test_num = 0; test_num < num_tests; ++test_num) { if (test_num % ten_percent == 0) { std::cout << "." << std::flush; } } auto end...

2021-11-29 14:35:00 369

原创 bert tokenizer

bert包括三个tokenizer：FullTokenizer，BasicTokenizer，WordpieceTokenizer，其中FullTokenizer就是调用后面两个bert tokenizer is not actually suitable for Chinese (and we don't include code to learn WordPieces), but the Google SentencePiece toolkit does have good support for

2021-11-27 13:45:36 1239

原创 pytorch TorchScript

model = BertClassifier() model.load_state_dict(torch.load('model.pth', map_location=torch.device('cuda' if torch.cuda.is_available() else 'cpu'))) model.eval() tokenizer = BertTokenizer.from_pretrained('hfl/chinese-roberta-wwm-ext') token = toke.

2021-11-21 18:01:56 805

原创 tensorflow模型转化为pb格式

def serving_input_fn(): label_ids = tf.placeholder(tf.int32, [None, 3], name='label_ids') input_ids = tf.placeholder(tf.int32, [None, 200], name='input_ids') input_mask = tf.placeholder(tf.int32, [None, 200], name='input_mask') segment_...

2021-11-21 16:37:29 1552

原创 pytorch模型转换成tf

参考：pytorch模型转tensorflow的pb文件 - 知乎pip install tensorflow-gpu==2.2.0 -i https://pypi.tuna.tsinghua.edu.cn/simplepip uninstall onnx-tf==1.6tensorflow2.2.0+onnx-tf1.6生成单个pb文件，

2021-11-16 14:34:10 360

原创 python http

python2python -m SimpleHTTPServer [port]python3python -m http.server [port]接收方：wget IP:port/filename -O filename

2021-01-12 11:14:40 58

原创 go interface类型转换

target是一个[{}]的interface类型。fmt.Println("type:", reflect.TypeOf(tar))for _, i := range tar.([]interface{}) { v, ok := i.(map[string]interface{}) if ok { text := v["text"].(string) ...

2020-03-19 11:15:23 358

原创 rabbitmq安装

安装erlang参考https://github.com/rabbitmq/erlang-rpm, 新建 /etc/yum.repos.d/rabbitmq_erlang.repo，内容如下：[rabbitmq_erlang]name=rabbitmq_erlangbaseurl=https://packagecloud.io/rabbitmq/erlang/el/7/$basearc...

2020-01-08 19:12:14 103

原创 pytorch安装

1. 利用清华镜像源安装添加镜像源conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/conda confi...

2020-01-07 10:46:03 397 1

原创 horovod安装

1.安装nccl查看cuda版本cat /usr/local/cuda/version.txt，从https://developer.nvidia.com/nccl/nccl-download安装对应的包sudo yum updatesudo yum install libnccl-2.5.6-1+cuda10.0 libnccl-devel-2.5.6-1+cuda10.0 libnccl...

2019-12-20 10:51:35 1837

原创利用腾讯词向量计算相似度

利用腾讯词向量计算相似度欢迎使用Markdown编辑器新的改变功能快捷键合理的创建标题，有助于目录的生成如何改变文本的样式插入链接与图片如何插入一段漂亮的代码片生成一个适合你的列表创建一个表格设定内容居中、居左、居右SmartyPants创建一个自定义列表如何创建一个注脚注释也是必不可少的KaTeX数学公式新的甘特图功能，丰富你的文章UML 图表FLowchart流程图导出与导入导出导入欢迎使用...

2019-10-17 10:08:06 1765

原创 golang整型字符串转换

整型转字符串：strconv.Itoa(int(i)) (Fast)strconv.FormatUint、strconv.FormatInt(int64(i), 10) (Faster)Itoa is equivalent to FormatInt(int64(i), 10).参考：https://stackoverflow.com/questions/39442167/conve...

2019-10-17 10:01:37 460

原创 http压测工具wrk

https://github.com/wg/wrkgit clone https://github.com/giltene/wrk2.gitmake编译./wrk -t2 --latency -c100 -d100s -R10000 "http=encode(中文)"一般两个线程t2

2019-08-30 10:13:09 243

原创 golang unrecognized path

mkdir src/golang.org/x/cd src/golang.org/x/git clone https://github.com/golang/crypto.gitgit clone https://github.com/golang/net.gitgit clone https://github.com/golang/sys.gitgit clone https...

2019-04-26 10:48:58 171

原创 pytorch学习

安装：pip install torch torchvision -i https://pypi.tuna.tsinghua.edu.cn/simple运算：矩阵乘矩阵：A*B/torch.mul(A, B)/ A.mul(B) 矩阵乘向量：torch.mv(A, b)或者b = torch.rand((4,1)) 向量点乘：torch.dot(a, b)...

2019-04-18 12:59:41 220

原创 lucene tfidf

lucene tfidf score获取idfindexReader.docFreq(new Term(FIELD, “中国”))indexReader.maxDoc()tf Terms terms = indexReader.getTermVector(docID, TEXT_FIELD); TermsEnum termsEnum = terms.iter...

2019-03-20 15:42:34 375

原创 md5

def curlmd5(src): m = hashlib.md5() m.update(src.encode('UTF-8')) return m.hexdigest()

2019-03-05 17:25:28 1696

原创 tensorflow docker国内镜像

https://hub.daocloud.io/repos/4e686d90-5e24-40b1-8bc6-8616b82f8143

2019-03-05 17:11:20 2623

原创 bottle处理post json数据请求

客户端import requestsimport jsonheaders = {'content-type':'application/json'}url="http://127.0.0.1:8888/server/" data = { 'mid':"01", 'content':'happy'}r = requests.post(url, data=json....

2019-02-25 18:25:37 2590

原创 java解析xml

package xml.learn;import java.io.File;import javax.xml.parsers.SAXParser;import javax.xml.parsers.SAXParserFactory;import org.xml.sax.Attributes;import org.xml.sax.SAXException;import org.xml...

2019-02-22 10:46:44 60

转载 python测试时间

from timeit import timeitimport rereg = re.compile('look')def find(string, text): if string.find(text) > -1: passdef re_search(string): if reg.search(string): pa...

2019-02-20 20:44:28 488

原创 lucene学习

Document：Documents are the unit of indexing and search. A Document is a set of fields. http://lucene.apache.org/core/2_9_4/api/all/org/apache/lucene/document/Document.html#add(org.apache.lucene.docume...

2019-02-20 20:31:47 165

原创 fasttext使用

步骤数据格式：__label__美容最新款美容黑科技训练：./fasttext supervised -input training_set.txt -output model -lr 1.0 -epoch 25 -wordNgrams 3测试：./fasttext test model.bin data/test_set.txt./fasttext predic...

2019-02-17 17:38:46 111

原创 The accumulated size of entities is "50,000,001" that exceeded the "50,000,000" limit set by "FEATUR

https://stackoverflow.com/questions/42991043/error-xml-sax-saxparseexception-while-parsing-a-xml-file-using-wikixmlj

2019-02-13 14:44:25 367

原创写入csv

import csvimport sysimport codecsreload(sys)sys.setdefaultencoding('utf8')with open('out.csv', 'w') as fw: csv_writer = csv.writer(fw,delimiter=' ') with codecs.open(input, 'r', 'utf-8'...

2019-01-02 23:16:59 193

原创 java读取Resources下文件

InputStream path = this.getClass().getResourceAsStream("/data.txt");BufferedReader reader = new BufferedReader(new InputStreamReader(path));

2018-12-25 11:08:48 2942

原创 kmeans

class Kmeans(): def __init__(): self.k_dict_last = defaultdict(list) self.k_dict = defaultdict(list) self.k_node = [] def eclidDist(vec1, vec2): sum = 0 ...

2018-10-10 21:10:31 88

转载求开根号

二分查找、牛顿法求开根号#include<stdio.h>float sqr(float n, float r){ float mid = n / 2; float min, max; min = 0; max = n; while ((mid*mid - n > r) ||(mid*mid - n < -r)) { if (mid...

2018-09-15 17:08:59 1154

转载二叉树

二叉树节点路径求和参考 https://www.nowcoder.com/questionTerminal/840dd2dc4fbd4b2199cd48f2dadf930a另一种解法：整体思路就是递归的去检索，每当经过一个节点的时候，就把节点值放入vector中，当到达叶子节点时，判断此时vector中的和是否等于给定sum，等于的话表示这是一条符合条件的路径，打印该路径。否则清空ve...

2018-09-14 10:53:54 100

原创链表中的第k个

struct ListNode{ int val; struct ListNode *next; ListNode(int x): val(x), next(NULL){ }}class Solution{public: ListNode* FindKthToTail(ListNode* PListHead, unsigned int k){ if(N...

2018-09-14 10:45:58 123

原创 c++ vector

#include <iostream>#include <vector>using namespace std;int main(){ vector<vector<int> > vv; //空格 vector<int> v; int tmp; for(int i=0;i<3;i++){ v.c...

2018-09-14 10:44:34 116

转载 leecode动态规划数组矩阵

１．LIS状态设计：F[i]代表以A[i]结尾的LIS的长度状态转移：F[i]=max{F[j]+1}(1<=j< i,A[j]< A[i])边界处理：F[i]=1(1<=i<=n)时间复杂度：O(n^2)int main(){ scanf("%d",&n); for(int i=1;i<=n;i++) { ...

2018-09-14 10:42:19 127

转载排序

快速排序void QuickSort(int r[], int low, int high){ int pos; if(low < high){ pos = partition(r, low, high); QuickSort(r, low, pos-1); QuickSort(r, pos+1, high); } }int partition...

2018-09-14 10:37:52 95

转载 leecode动态规划滑雪

１．滑雪//https://blog.csdn.net/qq_25202231/article/details/72925477// dp( i,j ) = Max( dp( i-1, j ), dp( i, j+1 ),dp( i+1, j ), dp( i, j-1 ) ) + 1;#include <iostream> #include <fstream&...

2018-09-14 10:35:03 223

空空如也

空空如也