2019 December 12 nginx, 上传, 下载

基于nginx的文件上传下载服务器

基于nginx的文件上传下载服务器

nginx本身可以作为静态资源下载服务器，经过扩展后nginx还可以作为上传服务器，本文将讨论如何使用nginx配置文件上传和下载服务器。

1. nginx下载配置

比如在88端口，启动下载服务器，可以下载 /home/download下的文件，配置如下：

server {
    listen       88;

    # download
    autoindex on;               # enable directory listing output
    autoindex_exact_size off;   # output file sizes rounded to kilobytes, megabytes, and gigabytes
    autoindex_localtime on;     # output local times in the directory

    location / {
        root /home/download;
    }
}

浏览器访问相关地址，可以得到如下结果。

# http://127.0.0.1:88

Index of /
../
download1.iso                                      14-Jan-2020 10:14       7
download2.iso                                      14-Jan-2020 10:14       0

2. nginx登录认证

在部分情况下，我们下载时需要对用户身份进行登录认证。我们可以使用 auth_basic_user_file 指定用户名密码文件，该文件我们可以使用如下方法生成：

echo -n 'admin:' >> .httppasswd
openssl passwd >> .httppasswd
# type your password twice

查看生成的密码文件：

# cat .httppasswd

admin:4spAtxv54N8aE

我们也可以openssl -arp1参数生成密码：

echo -n 'admin1:' >> .httppasswd
openssl passwd -apr1 >> .httppasswd
# type your password twice

查看生成的密码文件：

# cat .httppasswd

admin:4spAtxv54N8aE
admin1:$apr1$f/SE79g5$6WYahbVFFCZ0oxB1LZDJE/

其他openssl passwd的密码生成算法可以进一步阅读参考资料

密码文件生成后，可以在nginx配置中增加文件后重启nginx即可

server {
    listen       88;

    # auth
    auth_basic "Restricted site";
    auth_basic_user_file /etc/nginx/conf.d/.httppasswd;
    ......
}

此时我们访问相关页面就需要输入对应的用户名密码了。

3. nginx上传配置(lua-resty-upload)

lua-resty-upload是基于lua-resty-upload的lua文件上传处理代码。使用该方案依赖于nginx的扩展模块lua-nginx-module。

3.1 lua-nginx-module模块安装

  1. 下载安装LuaJIT 2.1：http://luajit.org/download.html
cd /usr/local/src
wget http://luajit.org/download/LuaJIT-2.1.0-beta2.tar.gz
tar zxf LuaJIT-2.1.0-beta2.tar.gz
cd LuaJIT-2.1.0-beta2
make PREFIX=/usr/local/luajit
make install PREFIX=/usr/local/luajit
  1. 下载ngx_devel_kit（NDK）模块 ：https://github.com/simpl/ngx_devel_kit/tags，不需要安装
cd /usr/local/src
wget https://github.com/simpl/ngx_devel_kit/archive/v0.2.19.tar.gz
tar -xzvf v0.2.19.tar.gz
  1. 下载最新的lua-nginx-module 模块 ：https://github.com/openresty/lua-nginx-module/tags，不需要安装
cd /usr/local/src
wget https://github.com/openresty/lua-nginx-module/archive/v0.10.15.tar.gz
tar -xzvf v0.10.15.tar.gz
  1. nginx编译增加模块

我们的nginx是使用yum安装的，可以下载对应版本的源码编译后替换

查看nginx版本和编译参数

# nginx -V

nginx version: nginx/1.16.1
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC)
built with OpenSSL 1.0.2k-fips  26 Jan 2017
TLS SNI support enabled
configure arguments: --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/run/nginx.pid --lock-path=/run/lock/subsys/nginx --user=nginx --group=nginx --with-file-aio --with-ipv6 --with-http_ssl_module --with-http_v2_module --with-http_realip_module --with-stream_ssl_preread_module --with-http_addition_module --with-http_xslt_module=dynamic --with-http_image_filter_module=dynamic --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_degradation_module --with-http_slice_module --with-http_stub_status_module --with-http_perl_module=dynamic --with-http_auth_request_module --with-mail=dynamic --with-mail_ssl_module --with-pcre --with-pcre-jit --with-stream=dynamic --with-stream_ssl_module --with-google_perftools_module --with-debug --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic' --with-ld-opt='-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E'

下载对应nginx版本

cd /usr/local/src
wget http://nginx.org/download/nginx-1.16.1.tar.gz
tar -xvzf nginx-1.16.1.tar.gz

设置LUA环境变量

export LUAJIT_LIB=/usr/local/luajit/lib
export LUAJIT_INC=/usr/local/luajit/include/luajit-2.1

增加参数重新编译

cd /usr/local/src/nginx-1.16.1

#编译./configure 后面加上 第1步nginx -V查看到的所有配置参数,后面加上你要加的模块配置
./configure --prefix=/usr/share/nginx ...（太长我省略了）  --add-module=/usr/local/src/nginx-upload-module

#千万别make install
make

测试和替换nginx

#验证新nginx是否可用验证编译后的nginx是否可以使用已有的配置
./objs/nginx -t

#停止nginx
nginx -s stop

#使用新nginx
#备份 
cp /usr/sbin/nginx /usr/sbin/nginx-bak
#替换 
cp ./objs/nginx /usr/sbin/nginx

#开启nginx
nginx

在nginx的配置文件中增加如下代码

location /hello_lua { 
      default_type 'text/plain'; 
      content_by_lua 'ngx.say("hello, lua")'; 
}

重启nginx后，浏览器访问相关页面地址

# 127.0.0.1:88/hello_lua

hello, lua

出现 hello, lua 则代表安装成功。

3.2 lua-resty-upload配置

下载upload.lua文件并放在相关目录：

#下载github项目
git clone https://github.com/openresty/lua-resty-upload.git

#拷贝配置文件
mkdir /usr/local/luajit/lib/lua/5.1/resty/
cp lua-resty-upload/lib/resty/upload.lua /usr/local/luajit/lib/lua/5.1/resty/

#给予相关权限，不然使用可能会报错
chmod -R 777 /usr/local/luajit/lib/lua/5.1/

3.3 nginx上传配置

lua_package_path '/usr/local/luajit/lib/lua/5.1/?.lua;;';
server {
    listen       88;

    # auth
    auth_basic "Restricted site";
    auth_basic_user_file /etc/nginx/conf.d/.httppasswd;

    # download
    autoindex on;               # enable directory listing output
    autoindex_exact_size off;   # output file sizes rounded to kilobytes, megabytes, and gigabytes
    autoindex_localtime on;     # output local times in the directory

    location / {
        root /home/download;
    }
    
    # upload
    location ~ ^/upload(/.*)?$ {
        set $store_path /home/download$1/;
        content_by_lua_file /usr/local/luajit/lib/lua/5.1/my_upload.lua;
    }

    # lua test
    location /hello_lua { 
        default_type 'text/plain'; 
        content_by_lua 'ngx.say("hello, lua")'; 
    }
}

3.4 my_upload.lua配置

local upload = require "resty.upload"

local function my_get_file_name(header)
    local file_name
    for i, ele in ipairs(header) do
        file_name = string.match(ele, 'filename="(.*)"')
        if file_name and file_name ~= '' then
            return file_name
        end
    end
    return nil
end

local chunk_size = 4096
local form = upload:new(chunk_size)
local file
local file_path
while true do
    local typ, res, err = form:read()

    if not typ then
         ngx.say("failed to read: ", err)
         return
    end

    if typ == "header" then
        local file_name = my_get_file_name(res)
        if file_name then
            file_path = ngx.var.store_path..file_name
            file = io.open(file_path, "w+")
            if not file then
                ngx.say("failed to open file ", file_path)
                return
            end
        end

    elseif typ == "body" then
        if file then
            file:write(res)
        end

    elseif typ == "part_end" then
        if file then
            file:close()
            file = nil
            ngx.say("upload to "..file_path.." successfully!")
        end
    elseif typ == "eof" then
        break

    else
        -- do nothing
    end
end

3.5 上传测试

curl -H "Authorization: Basic YWRtaW46YWRtaW4=" -F filea=@/opt/hello.txt http://127.0.0.1:88/upload
upload to /home/download//hello.txt successfully!

# 其中 YWRtaW46YWRtaW4= 为 user:password 的base64编码，如有兴趣可以进一步阅读参考资料

浏览器打开下载页面，文件已正确上传

# http://127.0.0.1:88

Index of /
../
download1.iso                                      14-Jan-2020 10:14       7
download2.iso                                      14-Jan-2020 10:14       0
hello.txt                                          19-Jan-2020 17:23     140

参考资料

  1. openssl passwd, https://blog.csdn.net/aixiaoxiaoyu/article/details/80485781
  2. NGINX as a file server, https://www.yanxurui.cc/posts/server/2017-03-21-NGINX-as-a-file-server/
  3. openresty/lua-resty-upload, https://github.com/openresty/lua-resty-upload
  4. Nginx安装lua-nginx-module模块, https://www.cnblogs.com/felixzh/p/8709201.html
  5. HTTP Authorization 之 Basic Auth, https://www.jianshu.com/p/4cd42f7359f4

All content is licensed under CC BY-NC-SA

Buit with Jekyll and 3-Jekyll theme • Hosted on Github

Table of Contents