Category Archives: Uncategorized

Python Src – How PyMem_GetAllocator() and PyMem_SetAllocator() work

_PyMem_RawMalloc is calling ‘malloc’ to allocate memory and return pointer
so _PyMem_RawMalloc is equal to ‘malloc’

1
2
3
4
5
6
7
8
9
10
11
static void *
_PyMem_RawMalloc(void *ctx, size_t size)
{
    /* PyMem_RawMalloc(0) means malloc(1). Some systems would return NULL
       for malloc(0), which would be treated as an error. Some platforms would
       return a pointer with no memory behind it, which would break pymalloc.
       To solve these problems, allocate an extra byte. */
    if (size == 0)
        size = 1;
    return malloc(size);
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
typedef struct {
    /* user context passed as the first argument to the 4 functions */
    void *ctx;
 
    /* allocate a memory block */
    void* (*malloc) (void *ctx, size_t size);
 
    /* allocate a memory block initialized by zeros */
    void* (*calloc) (void *ctx, size_t nelem, size_t elsize);
 
    /* allocate or resize a memory block */
    void* (*realloc) (void *ctx, void *ptr, size_t new_size);
 
    /* release a memory block */
    void (*free) (void *ctx, void *ptr);
} PyMemAllocatorEx;
 
#define PYRAW_FUNCS _PyMem_RawMalloc, _PyMem_RawCalloc, _PyMem_RawRealloc, _PyMem_RawFree // this means
//*ctx = NULL and (*malloc) = _PyMem_RawMalloc = malloc function in c
 
        PyMemAllocatorEx alloc = {NULL, PYRAW_FUNCS}; 
 
// alloc will be assigned to _PyMem_Raw, _PyMem, _PyObject
// for PyMem_GetAllocator, it will get alloc by domain name and pick up one of allocator _PyMem_Raw, _PyMem, _PyObject
        PyMem_SetAllocator(PYMEM_DOMAIN_RAW, &alloc);
        PyMem_SetAllocator(PYMEM_DOMAIN_MEM, &alloc);
        PyMem_SetAllocator(PYMEM_DOMAIN_OBJ, &alloc);
void
PyMem_SetAllocator(PyMemAllocatorDomain domain, PyMemAllocatorEx *allocator)
{
    switch(domain)
    {
    case PYMEM_DOMAIN_RAW: _PyMem_Raw = *allocator; break;
    case PYMEM_DOMAIN_MEM: _PyMem = *allocator; break;
    case PYMEM_DOMAIN_OBJ: _PyObject = *allocator; break;
    /* ignore unknown domain */
    }
}

There are 3 Domains that are PYMEM_DOMAIN_RAW, PYMEM_DOMAIN_MEM, PYMEM_DOMAIN_OBJ

alloc will be assigned to _PyMem_Raw, _PyMem, _PyObject based on 3 Domains.
for PyMem_GetAllocator, it will get alloc by domain name and pick up one of allocator _PyMem_Raw, _PyMem, _PyObject

Nginx – reverse proxy as load-balance in ubuntu

1. install nginx in ubuntu

1
$ sudo apt install ngnix

2. print ngnix conf path

1
$ sudo ngnix -t

3. create node.conf, in nginx.conf, it would load all file matched *.conf under conf.d/

1
2
$ touch /etc/nginx/conf.d/
$ vim /etc/nginx/conf.d/node.conf

4. node.conf file context.
it is best to keep X-Real-IP and X-Forwarded-For, because in node.js app, it would record client’s ip address into log

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
upstream nodeserver {
   server localhost:3085;
}
 
server {
   listen 8080;
   server_name linode.emacslisp.com;
 
   location /test {
       proxy_pass http://nodeserver;   
       proxy_set_header Host $host;
       proxy_set_header X-Real-IP $remote_addr;
       proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
       proxy_redirect     off;
       proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
       proxy_max_temp_file_size 0;
       proxy_connect_timeout      90;
       proxy_send_timeout         90;
       proxy_read_timeout         90;
       proxy_buffer_size          4k;
       proxy_buffers              4 32k;
       proxy_busy_buffers_size    64k;
       proxy_temp_file_write_size 64k;       
   }
}

javascript – download Blob file directly

As we know that for security issue, browser would disable javascript to download file directly.

but today what I want to show is to use javascript to click on ‘a’ link to download file
using Blob

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
  var blob = new Blob([csvFile], { type: 'text/csv;charset=utf-8;' });
  if (navigator.msSaveBlob) { // IE 10+
    navigator.msSaveBlob(blob, filename);
  } else {
    var link = document.createElement("a");
    if (link.download !== undefined) { 
      var url = URL.createObjectURL(blob);
      link.setAttribute("href", url);
      link.setAttribute("download", filename);
      link.style.visibility = 'hidden';
      document.body.appendChild(link);
      link.click();
      document.body.removeChild(link);
    }
  }

javascript – using await to make sleep

In old day, when using javascript to create timer,

setInterval or setTimeout are standard function to implement
A few other way to do is using third-party npm library.
or even calling c++ as native timer.

with ES2016, await and asyn keyword has been introducted into.
Here is example.

1
2
3
4
5
6
7
8
9
10
11
12
 
function sleep(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}
 
async function demo() {
  console.log('Taking a break...');
  await sleep(2000);
  console.log('Two second later');
}
 
demo();

VPS – add Swap to enhance performance

VPS normally does not have any swap space.

1
2
3
4
5
6
7
8
9
10
sudo /bin/dd if=/dev/zero of=/var/swap.1 bs=1M count=1024
sudo /sbin/mkswap /var/swap.1
sudo chmod 600 /var/swap.1
sudo /sbin/swapon /var/swap.1
 
/etc/fstab:
/var/swap.1 swap swap defaults 0 0
 
# if it not mounted as swap, using command 
sudo /sbin/swapon /var/swap.1

swap size should be 2 times than Memory for 512MB, 1GB Memory.
Why? take Memory size 512MB for example, when 512MB is full, it will put into some of data into swap,
if now machine is sleeping, it will put 512MB into swap.
therefore, ideally swap should be 2 times than physical memory.

However, for large memory such as 16GB, swap size is based on what you need, for example, 2GB.

Maven – different between -DskipTests and -Dmaven.test.skip=true

sometime when we build maven, junit will also be invoked to test code.

usually, these unit test will fail because of development machine configures are different.

there are two commands which could skip test.

1
2
$ mvn package -DskipTests
$ mvn package -Dmaven.test.skip=true

“-DskipTests” is to build test source code but don’t run them.
“-Dmaven.test.skip=true” don’t do anything for test source code, do not compile and do not run for test code.

google chart api for marker pin

In our project on google map, we have to display some custom marker pins use png.

google api provide great function for this purposes.
Here is a example URL

https://chart.googleapis.com/chart?chst=d_map_spin&chld=0.3|0|78909A|13|b|

you could change scale 0.3 to any size with fill in color 78909A,
font size 13 is for font to put into pin text

click on the url to view excellent markers.

radgrid

var grid = $find(“<%=RadGrid1.ClientID %>“);
var ctrl_hiddenfield = $telerik.findControl(grid.get_element(),”hiddenfield_name”);

http://docs.telerik.com/devtools/aspnet-ajax/controls/listbox/client-side-programming/overview

libtar and its api

libtar is small but effective library, although it has some bugs when dealing with long path.

The Steps are following:
download libtar-1.2.11 and uncompress it, the main path is /path/libtar-1.2.11

build with command “./configure && make”, if you want to debug, add “-g” for gcc to build.

make “test” folder “/path/libtar-1.2.11/test”. find header file and libtar.a file

put 1.c in “/path/libtar-1.2.11/test”

so finally, the build command will be “g++ 1.c -o 1 ../lib/libtar.a -I ../bin/libtar-1.2.11/include”

some important code, for example, char* subfile1 = “./sub1/1.txt”
this is for file under sub folder.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
 #include <tar.h>  
 #include <libtar.h>  
 #include <iostream>                                                             
 #include <fcntl.h>    
 
 using namespace std;                               
 
 int main(){                                                        
     // tar                         
     if(1){                                                          
         TAR* tar_handle;
         char* tar_fname = "test.tar";
         tar_open(&amp;tar_handle, tar_fname, NULL,  O_WRONLY | O_CREAT,  0644,  TAR_GNU);
         char* subfile1 = "./sub1/1.txt";
         tar_append_file(tar_handle, subfile1,  subfile1);
         char* subfile2 = "test1.c";
         tar_append_file(tar_handle, subfile2,  subfile2);
         tar_close(tar_handle);
     }
 
     // untar
     if(0){
         TAR* tar_handle;
         char* tar_fname = "test.tar";
         tar_open(&amp;tar_handle, tar_fname, NULL,  O_RDONLY,  0644,  TAR_GNU);
         char* savefold = "temp";
         tar_extract_all(tar_handle, "temp");
         tar_close(tar_handle);
     }
 
     return 0;
 }

example of jquery bxslider

Here is a full example of how to use bxslider to show image slider.

bxslider is also very good at fade in/out transition in div

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
<html>
  <title></title>
  <head>
    <!-- jQuery library (served from Google) -->
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.8.2/jquery.min.js"></script>
<!-- bxSlider Javascript file -->
<script src="jquery.bxslider.min.js"></script>
<!-- bxSlider CSS file -->
<link href="jquery.bxslider.css" rel="stylesheet" />
 
<script language="javascript">
$(document).ready(function(){
$('.bxslider').bxSlider({
  mode: 'fade',
  captions: true
});
});
</script>
  </head>
 
  <body>
    <ul class="bxslider">
  <li><img src="pic1.jpg" /></li>
  <li><img src="pic2.jpg" /></li>
  <li><img src="pic3.jpg" /></li>
</ul>
  </body>
</html>