Nuclio Deploying Functions

Published by onesixx on

Builtin Model 모델 배포

Serverless tutorial by CVAThttps://opencv.github.io/cvat/docs/manual/advanced/serverless-tutorial/

CVAT에서 이미 만들어진 (serverless) Function을 바로 배포만 하면된다.

cvat 프로젝트를 생성

여기에 serverless function과 model을 저장

Shell

Deploy function

CVAT에서 제공하는 배포가능한 serverless function 및 모델 ( https://github.com/openvinotoolkit/cvat/tree/develop/serverless)

Shell

로컬 환경에서 배포하는 경우에는 —-platform local 옵션을 추가
–http-trigger-service-type NodePort

Shell
Shell
Shell

Example using Python

https://nuclio.io/docs/latest/examples/#python-examples
Nuclio==>https://github.com/nuclio/nuclio/tree/master/hack/examples#python-examples
Hello World helloworldA simple function that
showcases unstructured logging and a structured response.
EncryptencryptA function that
uses a 3rd-party Python package to encrypt the event body, and showcases build commands for installing both OS-level and Python packages.
FaceRecognizerfaceA function that
uses Microsoft’s face API, configured with function environment variables.
The function uses 3rd-party Python packages, which are installed by using an inline configuration.
Sentiment Analysis sentimentsA function that
uses the vaderSentiment library to classify text strings into a negative or positive sentiment score.
TensorFlowtensorflowA function that
uses the inception model of the TensorFlow open-source machine-learning library to classify images.
The function demonstrates advanced uses of Nuclio with a custom base image, third-party Python packages, pre-loading data into function memory (the AI Model), structured logging, and exception handling.

Hello World

Function-Configuration

function.yaml, main.py, model_handler.py 준비

https://github.com/nuclio/nuclio/blob/development/docs/reference/function-configuration/function-configuration-reference.md

YAML/YML
function.yaml
Python
main.py
Python
model_handler.py

deploy functions

Shell
Shell
YAML/YML

모델배포 : custom model – detectron2

https://opencv.github.io/cvat/docs/manual/advanced/serverless-tutorial/#adding-your-own-dl-models

detectron2설치후 Local에서 DL모델 돌려보기

Shell
DETECTRON2
Python
/demo/demo.py 수정

function.yaml

1) Annotation 위해 CVAT이 활용할 serverless function 준비

https://opencv.github.io/cvat/docs/manual/advanced/serverless-tutorial/#dl-model-as-a-serverless-function

함수의 이름 정하기

파라미터 전체설명 : https://nuclio.io/docs/latest/reference/function-configuration/function-configuration-reference/

metadata.name : pth.facebookresearch.detectron2.retinanet_r101

metadata.annotations.name : 보여지는 이름

meradata.annotation.type: serverless function의 종류 . detector

metadata.annotation.framework: 기냥 정보성.. 특별히 정해진건 없으나..OpenVINO, PyTorch, TensorFlow, etc.

function.yaml

YAML/YML
function.yaml

외부에서 해당 모델을 통해 접근을 위한 설정

nuclio function의 포트번호와 cvat network를 지정

spec.triggers.myHttpTrigger.attributes에 특정 port를 추가하고, 
spec.platform.attributes 에 도커 컴포즈를 실행했을 때 생성된 도커 네트워크(cvat_cvat)를 추가.

YAML/YML
function.yaml

2. 로컬에서 DL모델을 실행하기 위한 소스코드를 Nuclio 플랫폼에 적용

2-1 모델을 메모리에 로딩 (init_context(context)함수를 사용하여)

https://nuclio.io/docs/latest/concepts/best-practices-and-common-pitfalls/#use-init_context-instead-of-global-variable-declarations-or-function-calls

Python
init_context

2-2 아래 프로세스를 위해 handler에 entry point를 정의하고, main.py에 넣는다.

  • accept incoming HTTP requests
  • run inference
  • reply with detection results

main.py

Python
main.py

3. deploy

새로운 serverless 함수를 사용하기 위해서는
(위 Builtin model에서 했던것처럼) nuctl 명령어로 deploy를 해야한다.

  • function.yaml
  • main.py
  • model_handler.py

방법1)

Shell

방법2)

Shell

Issue

https://github.com/opencv/cvat/issues/3457 : Steps for custom model deployment

http://How to upload DL which built by myself? : How to upload DL which built by myself?

https://github.com/opencv/cvat/issues/5551 : Mmdetection MaskRCNN serverless support for semi-automatic annotation

https://github.com/opencv/cvat/issues/4909: Load my own Yolov5 model on cvat by nuclio

mmdetection 모델변환

https://mmdetection.readthedocs.io/en/latest/useful_tools.html#model-conversion

Exporting MMDetection models to ONNX format
https://medium.com/axinc-ai/exporting-mmdetection-models-to-onnx-format-3ec839c38ff

openvino : https://da2so.tistory.com/63

Serving pre-trained ML/DL models
https://docs.mlrun.org/en/stable/tutorial/03-model-serving.html#serving-pre-trained-ml-dl-models

Categories: vision

onesixx

Blog Owner

Subscribe
Notify of
guest

0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x