{"id":29566,"date":"2024-10-28T01:55:29","date_gmt":"2024-10-28T01:55:29","guid":{"rendered":"http:\/\/atmokpo.com\/w\/?p=29566"},"modified":"2024-11-26T06:52:15","modified_gmt":"2024-11-26T06:52:15","slug":"%ed%97%88%ea%b9%85%ed%8e%98%ec%9d%b4%ec%8a%a4-%ed%8a%b8%eb%a0%8c%ec%8a%a4%ed%8f%ac%eb%a8%b8-%ed%99%9c%ec%9a%a9%ea%b0%95%ec%a2%8c-clip-%ea%b8%b0%eb%b0%98-%ec%82%ac%ec%a0%84%ed%95%99%ec%8a%b5-2","status":"publish","type":"post","link":"https:\/\/atmokpo.com\/w\/29566\/","title":{"rendered":"\ud5c8\uae45\ud398\uc774\uc2a4 \ud2b8\ub80c\uc2a4\ud3ec\uba38 \ud65c\uc6a9\uac15\uc88c, CLIP \uae30\ubc18 \uc0ac\uc804\ud559\uc2b5 \ubaa8\ub378 \uc2e0\uacbd\ub9dd \uad6c\uc870"},"content":{"rendered":"

<\/p>\n

\ub525\ub7ec\ub2dd\uc758 \ubc1c\uc804\uacfc \ud568\uaed8 \ub2e4\uc591\ud55c \uc790\uc5f0\uc5b4 \ucc98\ub9ac(NLP) \ubc0f \ucef4\ud4e8\ud130 \ube44\uc804(CV) \uae30\uc220\uc774 \uc735\ud569\ub418\uace0 \uc788\uc2b5\ub2c8\ub2e4. \uadf8 \uc911 \ud558\ub098\uc778 CLIP(Contrastive Language-Image Pre-training)\ub294 \uc774\ubbf8\uc9c0\uc640 \ud14d\uc2a4\ud2b8 \ub370\uc774\ud130\ub97c \ub3d9\uc2dc\uc5d0 \ub2e4\ub8e8\uc5b4 \uc5b8\uc5b4\uc640 \uc2dc\uac01 \uc815\ubcf4\ub97c \uc77c\uad00\ub418\uac8c \ucc98\ub9ac\ud558\ub294 \ubaa8\ub378\uc785\ub2c8\ub2e4. \uc774\ubc88 \uac15\uc88c\uc5d0\uc11c\ub294 CLIP \ubaa8\ub378\uc758 \uae30\ucd08\ubd80\ud130 \ud65c\uc6a9\uae4c\uc9c0 \uc2ec\ub3c4 \uc788\uac8c \ub2e4\ub904\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.<\/p>\n

1. CLIP \ubaa8\ub378 \uc18c\uac1c<\/h2>\n

CLIP\uc740 OpenAI\uc5d0 \uc758\ud574 \uac1c\ubc1c\ub41c \ubaa8\ub378\ub85c, \ub2e4\uc591\ud55c \uc774\ubbf8\uc9c0\uc640 \uadf8\uc5d0 \ud574\ub2f9\ud558\ub294 \uc124\uba85\uc774 \uc788\ub294 \ub370\uc774\ud130\ub97c \ud65c\uc6a9\ud558\uc5ec \uc0ac\uc804 \ud6c8\ub828\ub41c \ubaa8\ub378\uc785\ub2c8\ub2e4. \uc774 \ubaa8\ub378\uc740 \uc774\ubbf8\uc9c0\uc640 \ud14d\uc2a4\ud2b8 \uac04\uc758 \uad00\uacc4\ub97c \ud559\uc2b5\ud558\uc5ec \ud2b9\uc815 \ud14d\uc2a4\ud2b8 \uc124\uba85\uc5d0 \uac00\uc7a5 \uc801\ud569\ud55c \uc774\ubbf8\uc9c0\ub97c \ucc3e\uac70\ub098, \ubc18\ub300\ub85c \uc774\ubbf8\uc9c0\ub97c \uc124\uba85\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n

1.1 \uae30\ubcf8 \uc544\uc774\ub514\uc5b4<\/h3>\n

CLIP\uc758 \uae30\ubcf8 \uc544\uc774\ub514\uc5b4\ub294 \ucee8\ud2b8\ub77c\uc2a4\ud2f0\ube0c \ud559\uc2b5 \ubc29\uc2dd\uc73c\ub85c, \uc774\ubbf8\uc9c0\uc640 \ud14d\uc2a4\ud2b8 \uc30d\uc774 \uc62c\ubc14\ub974\uac8c \ub9e4\uce6d\ub418\ub3c4\ub85d \ud559\uc2b5\ud558\ub294 \uac83\uc785\ub2c8\ub2e4. \uc774\ub97c \ud1b5\ud574 \ubaa8\ub378\uc740 \ub2e4\uc591\ud55c \ube44\uc8fc\uc5bc\uacfc \uc5b8\uc5b4\uc801 \ud328\ud134\uc744 \ud568\uaed8 \uc774\ud574\ud560 \uc218 \uc788\ub294 \ub2a5\ub825\uc744 \uac00\uc9c0\uac8c \ub429\ub2c8\ub2e4.<\/p>\n

1.2 \uc0ac\uc804\ud559\uc2b5\uacfc \ud30c\uc778\ud29c\ub2dd<\/h3>\n

CLIP \ubaa8\ub378\uc740 \ub300\ub7c9\uc758 \uc774\ubbf8\uc9c0-\ud14d\uc2a4\ud2b8 \uc30d \ub370\uc774\ud130\ub85c \uc0ac\uc804 \ud6c8\ub828\uc744 \uc218\ud589\ud569\ub2c8\ub2e4. \uc774 \ud6c4, \ud2b9\uc815 \uc791\uc5c5\uc5d0 \ub9de\uac8c \ud30c\uc778\ud29c\ub2dd\ud558\uc5ec \uc751\uc6a9\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n

2. CLIP \ubaa8\ub378 \uc544\ud0a4\ud14d\ucc98<\/h2>\n

CLIP \ubaa8\ub378\uc740 \ud06c\uac8c \ub450 \uac00\uc9c0 \uad6c\uc131 \uc694\uc18c\ub85c \ub098\ub20c \uc218 \uc788\uc2b5\ub2c8\ub2e4. \ud558\ub098\ub294 \uc774\ubbf8\uc9c0 \uc778\ucf54\ub354\uc774\uba70, \ub2e4\ub978 \ud558\ub098\ub294 \ud14d\uc2a4\ud2b8 \uc778\ucf54\ub354\uc785\ub2c8\ub2e4. \uc774\ub4e4 \uac01\uac01\uc758 \uad6c\uc131 \uc694\uc18c\ub4e4\uc740 \ud14d\uc2a4\ud2b8\uc640 \uc774\ubbf8\uc9c0\ub97c \ubca1\ud130 \uacf5\uac04\uc5d0\uc11c \uc77c\uce58\uc2dc\ud0ac \uc218 \uc788\ub294 \ubc29\ubc95\uc73c\ub85c \ud568\uaed8 \uc791\ub3d9\ud569\ub2c8\ub2e4.<\/p>\n

2.1 \uc774\ubbf8\uc9c0 \uc778\ucf54\ub354<\/h3>\n

\uc774\ubbf8\uc9c0 \uc778\ucf54\ub354\ub294 \ube44\uc804 \ud2b8\ub79c\uc2a4\ud3ec\uba38(ViT) \ub610\ub294 CNN(Convolutional Neural Network) \uc544\ud0a4\ud14d\ucc98\ub97c \ud1b5\ud574 \uc774\ubbf8\uc9c0\ub97c \ubca1\ud130\ub85c \ubcc0\ud658\ud569\ub2c8\ub2e4.<\/p>\n

2.2 \ud14d\uc2a4\ud2b8 \uc778\ucf54\ub354<\/h3>\n

\ud14d\uc2a4\ud2b8 \uc778\ucf54\ub354\ub294 \uc77c\ubc18\uc801\uc73c\ub85c \ud2b8\ub79c\uc2a4\ud3ec\uba38 \uc544\ud0a4\ud14d\ucc98\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc785\ub825\ub41c \ud14d\uc2a4\ud2b8\ub97c \ubca1\ud130\ub85c \ubcc0\ud658\ud569\ub2c8\ub2e4.<\/p>\n

3. CLIP \ubaa8\ub378 \uc124\uce58\uc640 \uae30\ubcf8 \uc0ac\uc6a9\ubc95<\/h2>\n

CLIP \ubaa8\ub378\uc744 \uc0ac\uc6a9\ud558\uae30 \uc704\ud574 Hugging Face Transformers \ub77c\uc774\ube0c\ub7ec\ub9ac\ub97c \uc124\uce58\ud574\uc57c \ud569\ub2c8\ub2e4. \uc544\ub798 \uba85\ub839\uc5b4\ub97c \ud1b5\ud574 \uc124\uce58\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4:<\/p>\n

pip install transformers<\/code><\/pre>\n
3.1 \ubaa8\ub378 \ubd88\ub7ec\uc624\uae30<\/h3>\n
\ubaa8\ub378\uc744 \ub2e4\uc74c\uacfc \uac19\uc774 \ubd88\ub7ec\uc62c \uc218 \uc788\uc2b5\ub2c8\ub2e4:<\/p>\n
from transformers import CLIPProcessor, CLIPModel\n\n# CLIP \ubaa8\ub378\uacfc \ud504\ub85c\uc138\uc11c\ub97c \ubd88\ub7ec\uc635\ub2c8\ub2e4.\nmodel = CLIPModel.from_pretrained(\"openai\/clip-vit-base-patch16\")\nprocessor = CLIPProcessor.from_pretrained(\"openai\/clip-vit-base-patch16\")<\/code><\/pre>\n
3.2 \uc774\ubbf8\uc9c0\uc640 \ud14d\uc2a4\ud2b8\uc758 \uc720\uc0ac\ub3c4 \ud3c9\uac00<\/h3>\n
\ub2e4\uc74c\uc740 \uc8fc\uc5b4\uc9c4 \uc774\ubbf8\uc9c0\uc640 \ud14d\uc2a4\ud2b8 \uac04\uc758 \uc720\uc0ac\ub3c4\ub97c \ud3c9\uac00\ud558\ub294 \ucf54\ub4dc\uc785\ub2c8\ub2e4:<\/p>\n
import torch\nfrom PIL import Image\n\n# \uc774\ubbf8\uc9c0\uc640 \ud14d\uc2a4\ud2b8 \ub85c\ub4dc\nimage = Image.open(\"path_to_your_image.jpg\")\ntext = [\"A picture of a cat\", \"A picture of a dog\"]\n\n# \uc774\ubbf8\uc9c0\uc640 \ud14d\uc2a4\ud2b8\ub97c \uc804\ucc98\ub9ac\ud569\ub2c8\ub2e4.\ninputs = processor(text=text, images=image, return_tensors=\"pt\", padding=True)\n\n# \ubaa8\ub378 \ucd9c\ub825\nwith torch.no_grad():\n    logits_per_image, logits_per_text = model(**inputs)\n\n# \uc720\uc0ac\ub3c4 \uacc4\uc0b0\nprobs = logits_per_image.softmax(dim=1)\nprint(\"Text probabilities:\", probs)  # \uac01 \ud14d\uc2a4\ud2b8 \uc124\uba85\uc758 \uc720\uc0ac\ub3c4 \ud655\ub960<\/code><\/pre>\n
4. CLIP\uc758 \uc751\uc6a9 \uc0ac\ub840<\/h2>\n
CLIP\uc740 \ub2e4\uc74c\uacfc \uac19\uc740 \ub2e4\uc591\ud55c \ubd84\uc57c\uc5d0 \uc751\uc6a9\ub420 \uc218 \uc788\uc2b5\ub2c8\ub2e4:<\/p>\n