Skip to content
#

s3

Here are 3,292 public repositories matching this topic...

sevospl
sevospl commented Nov 6, 2021

Hi. First of all, kudos to you for the VFS cache - it's really brilliant. I have one request though.

For a little background, as of now I'm using the following settings:

--vfs-cache-mode full \
  --vfs-cache-max-size 500G \
  --vfs-cache-max-age 5000h \
  --vfs-cache-poll-interval 5m \
  --vfs-read-ahead 2G \
  --bwlimit-file 32M \
  --vfs-read-chunk-size=64M \
  --vfs-read-chunk-s
seaweedfs

SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.

  • Updated Nov 12, 2021
  • Go
thanos
jspeedz
jspeedz commented Sep 5, 2019

What went wrong?

I'm getting deprecation warnings with openSSL encryption.

[2019/09/05 08:38:52][info] Using Encryptor::OpenSSL to encrypt the archive.
[2019/09/05 08:40:22][warn] Pipeline STDERR Messages:
[2019/09/05 08:40:22][warn] (Note: may be interleaved if multiple commands returned error messages)
[2019/09/05 08:40:22][warn]
[2019/09/05 08:40:22][warn] *** WARNING : depre

zhezh
zhezh commented Aug 25, 2021

Problem description

Hi, smart_open currently supports hdfs://, viewfs:// is also a hadoop file system which is similar to hdfs://.

I find that tf.io.gfile.GFile of tensorflow can support both hdfs:// and viewfs://, is it possible that smart_open add support for viewfs://?

Thanks~

Improve this page

Add a description, image, and links to the s3 topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the s3 topic, visit your repo's landing page and select "manage topics."

Learn more