Category Archives: Uncategorized

Java Stream – Map and FlatMap

Here is an example from book, it converts list of string into list of chars without duplicated. it is good example to show what different between map and flatmap. basically map will keep original structure of input element while flatmap won’t keep structure of input element.

MySQL Core Src – Lex/Parse SQL Statement

SQL lex files:

sql/sql_lex.h、sql/lex_token.h、sql/lex.h、sql/lex_symbol.h

sql/gen_lex_token.cc、sql/sql_lex.cc

SQL grammar parser file:

sql/sql_yacc.yy、sql/sql_yacc.cc、sql/sql_yacc.h

SQL hint grammar parser file:

sql/sql_hints.yy、sql/sql_hints.yy.cc

mysql use bison (lex/yacc) to token sql statement and parse it, take sql_yacc.yy for example, it is main file contains all parse sql rules.

Git – How to sync with two remote repos using git

git remote -v
git remote
git remote add pb https://github.com/paulboone/ticgit

$ git remote -v
origin https://github.com/schacon/ticgit (fetch)
origin https://github.com/schacon/ticgit (push)
pb https://github.com/paulboone/ticgit (fetch)
pb https://github.com/paulboone/ticgit (push)

$ git fetch pb
remote: Counting objects: 43, done.
remote: Compressing objects: 100% (36/36), done.
remote: Total 43 (delta 10), reused 31 (delta 5)
Unpacking objects: 100% (43/43), done.
From https://github.com/paulboone/ticgit
* [new branch] master -> pb/master
* [new branch] ticgit -> pb/ticgit

$ git checkout master

$ git merge upstream/master # important, merge fetch code and master

$ git push

wordpress – wpdb init and generate all database schema

$wpdb = new wpdb( DB_USER, DB_PASSWORD, DB_NAME, DB_HOST );

// very important, it will init all wpdb public/private attributes
// such as $wpdb->termmeta, without call this function , $wpdb->termmeta will be empty string.
wp_set_wpdb_vars();

// get all include user tables and blog tables
echo wp_get_db_schema(‘all’);

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157

CREATE TABLE wp_users (
  ID BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
  user_login VARCHAR(60) NOT NULL DEFAULT '',
  user_pass VARCHAR(255) NOT NULL DEFAULT '',
  user_nicename VARCHAR(50) NOT NULL DEFAULT '',
  user_email VARCHAR(100) NOT NULL DEFAULT '',
  user_url VARCHAR(100) NOT NULL DEFAULT '',
  user_registered datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
  user_activation_key VARCHAR(255) NOT NULL DEFAULT '',
  user_status INT(11) NOT NULL DEFAULT '0',
  display_name VARCHAR(250) NOT NULL DEFAULT '',
  PRIMARY KEY  (ID),
  KEY user_login_key (user_login),
  KEY user_nicename (user_nicename),
  KEY user_email (user_email)
) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci;
CREATE TABLE wp_usermeta (
  umeta_id BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
  user_id BIGINT(20) UNSIGNED NOT NULL DEFAULT '0',
  meta_key VARCHAR(255) DEFAULT NULL,
  meta_value longtext,
  PRIMARY KEY  (umeta_id),
  KEY user_id (user_id),
  KEY meta_key (meta_key(191))
) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci;
CREATE TABLE wp_termmeta (
  meta_id BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
  term_id BIGINT(20) UNSIGNED NOT NULL DEFAULT '0',
  meta_key VARCHAR(255) DEFAULT NULL,
  meta_value longtext,
  PRIMARY KEY  (meta_id),
  KEY term_id (term_id),
  KEY meta_key (meta_key(191))
) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci;
CREATE TABLE wp_terms (
 term_id BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
 name VARCHAR(200) NOT NULL DEFAULT '',
 slug VARCHAR(200) NOT NULL DEFAULT '',
 term_group BIGINT(10) NOT NULL DEFAULT 0,
 PRIMARY KEY  (term_id),
 KEY slug (slug(191)),
 KEY name (name(191))
) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci;
CREATE TABLE wp_term_taxonomy (
 term_taxonomy_id BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
 term_id BIGINT(20) UNSIGNED NOT NULL DEFAULT 0,
 taxonomy VARCHAR(32) NOT NULL DEFAULT '',
 description longtext NOT NULL,
 parent BIGINT(20) UNSIGNED NOT NULL DEFAULT 0,
 COUNT BIGINT(20) NOT NULL DEFAULT 0,
 PRIMARY KEY  (term_taxonomy_id),
 UNIQUE KEY term_id_taxonomy (term_id,taxonomy),
 KEY taxonomy (taxonomy)
) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci;
CREATE TABLE wp_term_relationships (
 object_id BIGINT(20) UNSIGNED NOT NULL DEFAULT 0,
 term_taxonomy_id BIGINT(20) UNSIGNED NOT NULL DEFAULT 0,
 term_order INT(11) NOT NULL DEFAULT 0,
 PRIMARY KEY  (object_id,term_taxonomy_id),
 KEY term_taxonomy_id (term_taxonomy_id)
) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci;
CREATE TABLE wp_commentmeta (
  meta_id BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
  comment_id BIGINT(20) UNSIGNED NOT NULL DEFAULT '0',
  meta_key VARCHAR(255) DEFAULT NULL,
  meta_value longtext,
  PRIMARY KEY  (meta_id),
  KEY comment_id (comment_id),
  KEY meta_key (meta_key(191))
) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci;
CREATE TABLE wp_comments (
  comment_ID BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
  comment_post_ID BIGINT(20) UNSIGNED NOT NULL DEFAULT '0',
  comment_author tinytext NOT NULL,
  comment_author_email VARCHAR(100) NOT NULL DEFAULT '',
  comment_author_url VARCHAR(200) NOT NULL DEFAULT '',
  comment_author_IP VARCHAR(100) NOT NULL DEFAULT '',
  comment_date datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
  comment_date_gmt datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
  comment_content text NOT NULL,
  comment_karma INT(11) NOT NULL DEFAULT '0',
  comment_approved VARCHAR(20) NOT NULL DEFAULT '1',
  comment_agent VARCHAR(255) NOT NULL DEFAULT '',
  comment_type VARCHAR(20) NOT NULL DEFAULT '',
  comment_parent BIGINT(20) UNSIGNED NOT NULL DEFAULT '0',
  user_id BIGINT(20) UNSIGNED NOT NULL DEFAULT '0',
  PRIMARY KEY  (comment_ID),
  KEY comment_post_ID (comment_post_ID),
  KEY comment_approved_date_gmt (comment_approved,comment_date_gmt),
  KEY comment_date_gmt (comment_date_gmt),
  KEY comment_parent (comment_parent),
  KEY comment_author_email (comment_author_email(10))
) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci;
CREATE TABLE wp_links (
  link_id BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
  link_url VARCHAR(255) NOT NULL DEFAULT '',
  link_name VARCHAR(255) NOT NULL DEFAULT '',
  link_image VARCHAR(255) NOT NULL DEFAULT '',
  link_target VARCHAR(25) NOT NULL DEFAULT '',
  link_description VARCHAR(255) NOT NULL DEFAULT '',
  link_visible VARCHAR(20) NOT NULL DEFAULT 'Y',
  link_owner BIGINT(20) UNSIGNED NOT NULL DEFAULT '1',
  link_rating INT(11) NOT NULL DEFAULT '0',
  link_updated datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
  link_rel VARCHAR(255) NOT NULL DEFAULT '',
  link_notes mediumtext NOT NULL,
  link_rss VARCHAR(255) NOT NULL DEFAULT '',
  PRIMARY KEY  (link_id),
  KEY link_visible (link_visible)
) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci;
CREATE TABLE wp_options (
  option_id BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
  option_name VARCHAR(191) NOT NULL DEFAULT '',
  option_value longtext NOT NULL,
  autoload VARCHAR(20) NOT NULL DEFAULT 'yes',
  PRIMARY KEY  (option_id),
  UNIQUE KEY option_name (option_name)
) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci;
CREATE TABLE wp_postmeta (
  meta_id BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
  post_id BIGINT(20) UNSIGNED NOT NULL DEFAULT '0',
  meta_key VARCHAR(255) DEFAULT NULL,
  meta_value longtext,
  PRIMARY KEY  (meta_id),
  KEY post_id (post_id),
  KEY meta_key (meta_key(191))
) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci;
CREATE TABLE wp_posts (
  ID BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
  post_author BIGINT(20) UNSIGNED NOT NULL DEFAULT '0',
  post_date datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
  post_date_gmt datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
  post_content longtext NOT NULL,
  post_title text NOT NULL,
  post_excerpt text NOT NULL,
  post_status VARCHAR(20) NOT NULL DEFAULT 'publish',
  comment_status VARCHAR(20) NOT NULL DEFAULT 'open',
  ping_status VARCHAR(20) NOT NULL DEFAULT 'open',
  post_password VARCHAR(255) NOT NULL DEFAULT '',
  post_name VARCHAR(200) NOT NULL DEFAULT '',
  to_ping text NOT NULL,
  pinged text NOT NULL,
  post_modified datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
  post_modified_gmt datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
  post_content_filtered longtext NOT NULL,
  post_parent BIGINT(20) UNSIGNED NOT NULL DEFAULT '0',
  guid VARCHAR(255) NOT NULL DEFAULT '',
  menu_order INT(11) NOT NULL DEFAULT '0',
  post_type VARCHAR(20) NOT NULL DEFAULT 'post',
  post_mime_type VARCHAR(100) NOT NULL DEFAULT '',
  comment_count BIGINT(20) NOT NULL DEFAULT '0',
  PRIMARY KEY  (ID),
  KEY post_name (post_name(191)),
  KEY type_status_date (post_type,post_status,post_date,ID),
  KEY post_parent (post_parent),
  KEY post_author (post_author)
) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci;

Java – long Value Mod Integer Value

Here is a java code for long % int (long mod int)

long v = 51;
long result = v%(10^9 + 7);

what is result value. it may looks to be surprised that result is 25.

result is supported to be 51.

here is right way to go.

long v = 51;
long result = v%100000007L
int finalResult = (int) result;

Idea here is very clear that we convert long%int to be long%long and convert result to be int.

Python Src – How PyMem_GetAllocator() and PyMem_SetAllocator() work

_PyMem_RawMalloc is calling ‘malloc’ to allocate memory and return pointer
so _PyMem_RawMalloc is equal to ‘malloc’

1
2
3
4
5
6
7
8
9
10
11

static void *
_PyMem_RawMalloc(void *ctx, size_t size)
{
    /* PyMem_RawMalloc(0) means malloc(1). Some systems would return NULL
       for malloc(0), which would be treated as an error. Some platforms would
       return a pointer with no memory behind it, which would break pymalloc.
       To solve these problems, allocate an extra byte. */
    if (size == 0)
        size = 1;
    return malloc(size);
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

typedef struct {
    /* user context passed as the first argument to the 4 functions */
    void *ctx;
 
    /* allocate a memory block */
    void* (*malloc) (void *ctx, size_t size);
 
    /* allocate a memory block initialized by zeros */
    void* (*calloc) (void *ctx, size_t nelem, size_t elsize);
 
    /* allocate or resize a memory block */
    void* (*realloc) (void *ctx, void *ptr, size_t new_size);
 
    /* release a memory block */
    void (*free) (void *ctx, void *ptr);
} PyMemAllocatorEx;
 
#define PYRAW_FUNCS _PyMem_RawMalloc, _PyMem_RawCalloc, _PyMem_RawRealloc, _PyMem_RawFree // this means
//*ctx = NULL and (*malloc) = _PyMem_RawMalloc = malloc function in c
 
        PyMemAllocatorEx alloc = {NULL, PYRAW_FUNCS}; 
 
// alloc will be assigned to _PyMem_Raw, _PyMem, _PyObject
// for PyMem_GetAllocator, it will get alloc by domain name and pick up one of allocator _PyMem_Raw, _PyMem, _PyObject
        PyMem_SetAllocator(PYMEM_DOMAIN_RAW, &alloc);
        PyMem_SetAllocator(PYMEM_DOMAIN_MEM, &alloc);
        PyMem_SetAllocator(PYMEM_DOMAIN_OBJ, &alloc);
void
PyMem_SetAllocator(PyMemAllocatorDomain domain, PyMemAllocatorEx *allocator)
{
    switch(domain)
    {
    case PYMEM_DOMAIN_RAW: _PyMem_Raw = *allocator; break;
    case PYMEM_DOMAIN_MEM: _PyMem = *allocator; break;
    case PYMEM_DOMAIN_OBJ: _PyObject = *allocator; break;
    /* ignore unknown domain */
    }
}

There are 3 Domains that are PYMEM_DOMAIN_RAW, PYMEM_DOMAIN_MEM, PYMEM_DOMAIN_OBJ

alloc will be assigned to _PyMem_Raw, _PyMem, _PyObject based on 3 Domains.
for PyMem_GetAllocator, it will get alloc by domain name and pick up one of allocator _PyMem_Raw, _PyMem, _PyObject

Nginx – reverse proxy as load-balance in ubuntu

1. install nginx in ubuntu

1

$ sudo apt install ngnix

2. print ngnix conf path

1

$ sudo ngnix -t

3. create node.conf, in nginx.conf, it would load all file matched *.conf under conf.d/

1
2

$ touch /etc/nginx/conf.d/
$ vim /etc/nginx/conf.d/node.conf

4. node.conf file context.
it is best to keep X-Real-IP and X-Forwarded-For, because in node.js app, it would record client’s ip address into log

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

upstream nodeserver {
   server localhost:3085;
}
 
server {
   listen 8080;
   server_name linode.emacslisp.com;
 
   location /test {
       proxy_pass http://nodeserver;   
       proxy_set_header Host $host;
       proxy_set_header X-Real-IP $remote_addr;
       proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
       proxy_redirect     off;
       proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
       proxy_max_temp_file_size 0;
       proxy_connect_timeout      90;
       proxy_send_timeout         90;
       proxy_read_timeout         90;
       proxy_buffer_size          4k;
       proxy_buffers              4 32k;
       proxy_busy_buffers_size    64k;
       proxy_temp_file_write_size 64k;       
   }
}

javascript – download Blob file directly

As we know that for security issue, browser would disable javascript to download file directly.

but today what I want to show is to use javascript to click on ‘a’ link to download file
using Blob

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

  var blob = new Blob([csvFile], { type: 'text/csv;charset=utf-8;' });
  if (navigator.msSaveBlob) { // IE 10+
    navigator.msSaveBlob(blob, filename);
  } else {
    var link = document.createElement("a");
    if (link.download !== undefined) { 
      var url = URL.createObjectURL(blob);
      link.setAttribute("href", url);
      link.setAttribute("download", filename);
      link.style.visibility = 'hidden';
      document.body.appendChild(link);
      link.click();
      document.body.removeChild(link);
    }
  }

javascript – using await to make sleep

In old day, when using javascript to create timer,

setInterval or setTimeout are standard function to implement
A few other way to do is using third-party npm library.
or even calling c++ as native timer.

with ES2016, await and asyn keyword has been introducted into.
Here is example.

1
2
3
4
5
6
7
8
9
10
11
12

 
function sleep(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}
 
async function demo() {
  console.log('Taking a break...');
  await sleep(2000);
  console.log('Two second later');
}
 
demo();

VPS – add Swap to enhance performance

VPS normally does not have any swap space.

1
2
3
4
5
6
7
8
9
10

sudo /bin/dd if=/dev/zero of=/var/swap.1 bs=1M count=1024
sudo /sbin/mkswap /var/swap.1
sudo chmod 600 /var/swap.1
sudo /sbin/swapon /var/swap.1
 
/etc/fstab:
/var/swap.1 swap swap defaults 0 0
 
# if it not mounted as swap, using command 
sudo /sbin/swapon /var/swap.1

swap size should be 2 times than Memory for 512MB, 1GB Memory.
Why? take Memory size 512MB for example, when 512MB is full, it will put into some of data into swap,
if now machine is sleeping, it will put 512MB into swap.
therefore, ideally swap should be 2 times than physical memory.

However, for large memory such as 16GB, swap size is based on what you need, for example, 2GB.